Mac OSX tuning is similar to FreeBSD.

sysctl -w net.inet.tcp.win_scale_factor=8
# set to at leaset 16M, and 32M for 10G hosts
sysctl -w kern.ipc.maxsockbuf=16777216
sysctl -w net.inet.tcp.sendspace=16777216
sysctl -w net.inet.tcp.recvspace=16777216

For OSX 10.4, Apple also provides a Broadband Tuner patch that does a less aggressive version of this for Mac OS X 10.4.

Starting with OSX 10.5, Apple added "self tuning TCP", and increased the default maximum socket buffer to 8MB, a much more reasonable value. Unfortunely they also reduced the default win_scale_factor to 3, which prevents the TCP window from ever openning up large enough for high BDP networks. Be sure to set net.inet.tcp.win_scale_factor as shown above.

More information can be found at this OSX Network Tuning Guide.