Building Linux PAM modules on OpenSolaris

Preface: I’m sure a good bit of this solution is sloppy, disorganized or just plain the wrong way to do things. I’m certainly welcoming suggestions from anyone more experienced at this. But hey, it works, which is more than I can say for most of the other stuff I’ve read on this, which was mostly aggregated from disorganized mailing lists and Sun forum postings.

I’ve gone through something of an ordeal trying to get pam_listfile working on OpenSolaris. pam_list is a reasonable Solaris-native approximation, but it only works for users and NIS netgroups — not Unix groups. Obviously this isn’t a huge problem if you’re using NIS, but it presents some difficulties if you’re using LDAP without maintaining NIS schema attributes. The only reasonable solution was to port the module I already knew from Linux PAM. This should be a simple and straightforward process. As it turns out, it’s not difficult, but it really isn’t straightforward at all, especially if you’re inexperienced in building software without getting automake to do all the heavy lifting for you.

All of the documentation I’ve found for this online seems to be wrong or missing steps. In building pam_listfile, here’s what I found to work:

• Download the Linux-PAM sources (I used 1.0.4) and extract them into ~.
• Configure your environment and set your $CC correctly. I used /opt/csw/bin/gcc4/gcc from Blastwave’s IPSgcc4core package. I don’t know if it will work with the Sun Studio compiler; I haven’t tested it.
• cd into ~/Linux-PAM-1.0.4 and run a ./configure. You’ll need the config.h it outputs in a little bit.
• The code as it exists won’t build on Solaris, because of differing definitions of getspnam_r and some other miscellaneous functions. Download this patch from Tim Small into ~/Linux-PAM-1.0.4, then patch -p1 < linux-pam-101.diff. This should fix most of the remaining Linuxisms in the core that will bite you.
• Build the support libraries. cd into ~/Linux-PAM-1.0.4/libpam and run make to generate .libs/*.o, which you’ll need later.
• If you don’t have a correctly-configured libiberty distributed with your GCC (as neither the Sun nor Blastwave packages do), edit libpam/pam_private.h and remove the reference to libiberty.h.
• If you don’t have a working libiberty, grab some vasprintf.c and asprintf.c sources from somewhere, since Sun doesn’t seem to implement them. You could probably get them out of the GCC sources, but I pulled them from an Apple public repository here and here. Stick them somewhere. I just dumped them in my ~/Linux-PAM-1.0.4/libpam directory. Compile them into object files with gcc -c asprintf.c. vasprintf.c.
• cd into your chosen module directory. For me, it was ~/Linux-PAM-1.0.4/modules/pam_listfile.
• Compile your PAM module into an object file: gcc -c -g -O2 -D_REENTRANT -DPAM_DYNAMIC -Wall -fPIC -I../../libpam/include -I../../libpamc/include -I../.. pam_listfile.c. (If the blog font makes this difficult, those are capitals of the letter “i” and not lowercase of the letter “L”.)
• Finally, statically link the PAM module. It won’t link to libpam.so by default, so you’ll need to do that manually. You’ll also need to statically link the pam_modutil stuff and the vasprintf/asprintf implementations you built, so /usr/ccs/bin/ld -o pam_listfile.so -l pam -B dynamic -G -z defs -lc pam_listfile.o ../../libpam/.libs/pam_syslog.o ../../libpam/.libs/pam_modutil_*.o ../../libpam/asprintf.o ../../libpam/vasprintf.o

Now you’ve got a shiny new pam_listfile.so file.

This may or may not work on Solaris 10. Let me know if you try it.