It was incredibly difficult to figure this out. I finally found a solution to allowing my OpenGL app to be anti-aliased in an OpenGL context within a Cocoa App’s window using Xcode.
In the end the steps are quite simple. Open you .xib that contains the OpenGL context. Select that container and open View > Utilities > Attributes Inspector
Then find the Sampling and Anti-Aliasing parameters and make sure they’re both set to something reasonable.