I also like to use CSS animations, not JS, but in such cases you have to constantly crutches, small or large. I know two ways to solve the problem. Usually, I use the first, because of its simplicity and speed of implementation. Although, of course, the second is more flexible and reliable.
1 - Simple and trite hardcode. If I make a CSS-animation of the disappearance of an element with a duration of 600ms, I postpone the deletion of an element in a timeout for an appropriate time. So in your case:
$confirm.addClass('hide'); setTimeout(function(){ $confirm.remove(); },600)
The problem is that time is set in two unrelated places. If you (or another prog after you) change the duration of the CSS-animation, for example, it is logical that this is enough - get an unexpected result.
2 - hang up on the element a listener of the event "transitionend" and perform actions at the end of the animation. The dependence on time is no longer and there is flexibility. But, in the case of two-way animation (as in the example, when an element can be at the end of the animation both on the left and on the right), you should also check the state of the element inside the callback in order to understand whether we are turned on or turned off.
Working example: https://jsfiddle.net/ipshenicyn/1xz9by4o/ Example of use:
$("#id").on('webkitTransitionEnd otransitionend oTransitionEnd msTransitionEnd transitionend', function(e){ //do semething })