As you may have noticed, freedesktop.org sort of got compromised a few days back. By 'sort of', I do, of course, mean 'totally'. Adam Conrad noticed a few thousand bounces in his inbox courtesy of being on www-data, and that they were all for spams being sent as www-data. Whoops. We started hunting for an insecure formmail.pl, but when we took a look at lsof and discovered an IRC proxy running, we decided it was something more insidious. From there, the machine got killed to all access but ours, and we started tracking down the point of entry. It turned out that it was compromised via a hole in TWiki, but no news was to be found on the TWiki site about this hole, nor was there a new release. How not to do security 101.[
At this point, we came to the conclusion that all we could do from here was
reinstall, so Keith got a call (from an Australian mobile, roaming into the UK,
to a US mobile; I fear to think how much that cost) letting him know the score.
Local muscle on site, we dug in and prepared for a reinstall. Most people
familiar with the freedesktop.org setup (and my writings on here about 'ill')
know that the setup was accumulated, not designed, and was horrifically out
of control. It was a mess, and probably incredibly insecure. Very few things
were done properly to scale to where it was. So, we took a deep breath, and
noted that this was a blessing in disguise as we got to sit back and have a
think about what we were doing this time. I got out some pieces of paper
and started scribbling (across six of them, actually), and we all got chatting
on what we could do when we rebuilt it all.
LDAP is already running on a separate machine, using Debian's userdir-ldap.
We have a separate source machine on our hitlist; hosting only CVS/SVN/Arch
repositories, and various web downloads. These downloads would have to be
signed for somehow, and all provided in a common download area. Three huge
hits: we're mirrorable, there's an audit trail and security on source, and
the general access machine and the source server are totally separate. Rock
SSH access is open to the general public, with the old home directories in
/home/compromised. If you administer a project with CVS or whatever, please
check that it hasn't been tainted. You can compare the repositories in
/cvs and /compromised-cvs to see the difference; /cvs contains the repositories
as they were on 15th Oct.
Administering your account requires a GPG key. Admins will be rather loathe
to perform menial duties (e.g. changing SSH keys) on a regular basis, so if
you ask us for anything, make sure it's to add a GPG key to your account.
This way, it's the same amount of work for us, and it ensures that you can
take care of your own account in future: less work for both of us, and less
time spent waiting.
Did I mention you should all have GnuPG keys? No, really. They're incredibly
useful. If we had signed copies of everything, verification would be an utter
doddle. But we don't, so it isn't.
Enjoy the new gabe.freedesktop.org. -daniels