Override NSWindow close button

Question

I'm working on a application for OS X where I use a custom window that draws an image as the background, including the title bar. I've been modifying this code to draw the window, and then calling [NSWindow standardWindowButton:forStyleMask:] to get the standard close, minimize, and maximize buttons.

The problem is my application uses NSPopovers, and when I close or minimize the application while a popover is open, it will close the popover or show an animation of the popover minimizing instead of closing the application. Is there a way to override the default close/minimize behavior in an NSWindow so I can dismiss any open popovers first?

Thanks, and sorry if this is an obvious question — this is my first time working with the OS X SDK so I don't have a lot of experience.

EDIT: A few hours after I posted this I thought I had an obvious solution – use the NSWindowDelegate methods "windowWillClose:" and "windowWillMiniaturize:" and dismiss the popovers there. However, it seems that since the close/minimize buttons are closing the popover, that if a popover is open, these delegate methods won't be called. This takes me back to step 1, but hopefully knowing the behavior will help someone figure out the problem.

There's also another problem in with the NSPopovers and I don't know if it's connected or not, so I thought I would add it here just in case there was a common cause. Sometimes, when I attempt to dismiss a popover, I'll get this error (for context, I'm pressing an NSButton that calls a function that checks for the existence of the popover, and if it exists, closes it):

2011-08-30 11:24:08.949 Playground[11194:707] *** Assertion failure in +[NSView _findFirstKeyViewInDirection:forKeyLoopGroupingView:], /SourceCache/AppKit/AppKit-1138/AppKit.subproj/NSView.m:11026
2011-08-30 11:24:08.950 Playground[11194:707] this method is supposed to only be invoked on top level items
2011-08-30 11:24:08.958 Playground[11194:707] (
     0   CoreFoundation                      0x00007fff873d4986 __exceptionPreprocess + 198
     1   libobjc.A.dylib                     0x00007fff87ac6d5e objc_exception_throw + 43
     2   CoreFoundation                      0x00007fff873d47ba +[NSException raise:format:arguments:] + 106
     3   Foundation                          0x00007fff8950314f -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 169
     4   AppKit                              0x00007fff88211064 +[NSView _findFirstKeyViewInDirection:forKeyLoopGroupingView:] + 137
     5   AppKit                              0x00007fff87d1f546 _replacementKeyViewAlongKeyViewPath + 565
     6   AppKit                              0x00007fff87d1f2ff -[NSView nextValidKeyView] + 179
     7   AppKit                              0x00007fff87d1f199 -[NSWindow _selectFirstKeyView] + 714
     8   AppKit                              0x00007fff882361cf _NSWindowRecursiveFindFirstResponder + 164
     9   AppKit                              0x00007fff882395c8 _NSWindowExchange + 79
     10  AppKit                              0x00007fff883a7e3a -[_NSWindowTransformAnimation startAnimation] + 426
     11  AppKit                              0x00007fff87c98bb2 -[NSWindow _doOrderWindow:relativeTo:findKey:forCounter:force:isModal:] + 592
     12  AppKit                              0x00007fff87c9890f -[NSWindow orderWindow:relativeTo:] + 154
     13  AppKit                              0x00007fff883dfaf0 _NSPopoverCloseAndAnimate + 948
     14  Playground                          0x00000001000078a4 -[MainWindowController dismissPopover:] + 100
     15  Playgorund                          0x0000000100007012 -[MainWindowController requestWasClicked:] + 98
     16  CoreFoundation                      0x00007fff873c411d -[NSObject performSelector:withObject:] + 61
     17  AppKit                              0x00007fff87ca2852 -[NSApplication sendAction:to:from:] + 139
     18  AppKit                              0x00007fff87ca2784 -[NSControl sendAction:to:] + 88
     19  AppKit                              0x00007fff87ca26af -[NSCell _sendActionFrom:] + 137
     20  AppKit                              0x00007fff87ca1b7a -[NSCell trackMouse:inRect:ofView:untilMouseUp:] + 2014
     21  AppKit                              0x00007fff87d2157c -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:] + 489
     22  AppKit                              0x00007fff87ca0786 -[NSControl mouseDown:] + 786
     23  AppKit                              0x00007fff87c6b66e -[NSWindow sendEvent:] + 6280
     24  AppKit                              0x00007fff87c03f19 -[NSApplication sendEvent:] + 5665
     25  AppKit                              0x00007fff87b9a42b -[NSApplication run] + 548
     26  AppKit                              0x00007fff87e1852a NSApplicationMain + 867
     27  Playground                          0x0000000100001c52 main + 34
     28  Playground                          0x0000000100001c24 start + 52
     29  ???                                 0x0000000000000001 0x0 + 1
)

Answer 1

The solution turned out to be fairly simple.

When I created the button on the NSWindow, I changed the action and target of the button:

[closeButton setTarget:self.delegate]; // alternatively you can make it self.windowController
[closeButton setAction:@selector(closeThisWindow:)]; 

And then in the NSWindowController subclass, I implemented the method:

-(void)closeThisWindow {
    [self close]; // for the minimize button you'll call [self.window miniaturize]
}

For some reason, the NSPopovers always reappear when the window is reopened or unminimized; since I actually want this behavior in my application, it's not a problem, but it's something to keep in mind if you use this solution. If you don't have any child windows, then you can just iterate through self.window.childWindows because NSPopovers are considered child windows. If you have other child windows you want to handle separately, you could add an array to the NSWindow subclass that monitors all of your popovers and just iterate through that.