SysAdmin Blog

Wow, currently I'm seriously underwhelmed by most of the presentations I've seen until now...

• Peter Glaser - Die Üblichen Verdächtigen (cache): 21C3 keynote (in german). Peter Glaser shows some differences between technocratic, amoral geeks like Wernher von Braun and the hacker culture
• Alexander Bernauer, Ansgar Wiechers - Unsicherheit von Personal Firewalls (cache): only saw the last few minutes of this one, but was not very impressed by the code examples shown. Hijacking IE to bypass personal firewalls is hardly news, but their infos (cache) have a few more interesting examples.
• Nils Magnus - SAP R/3 Protocol Reverse Engineering (cache): I expected this one to be quite interesting, but apart from a few general rants concerning host and network security in SAP environments, the presenter (who was 15 minutes late, too) didn't have anyting interesting. Obviously he started reverse-engineering some SAP protocol with a colleague (the SAP expert who wasn't present), but didn't get very far beyond some initial stage. Most moderately insightful comments came from the audience.
• starbug - Biometrie in Ausweisdokumenten (cache): Again, was quite late for this one, and only got some quite incoherent remarks about development in the legislative arena despite obvious technical problems with biometric methods.
• starbug - Biometrie �überwinden (cache): no way to get into this one, way overcowded (and unfortunately, two of the good ideas this year, a cool audio-streaming service via the DECT exchange and additional TVs in front of the lecture rooms, didn't really work out).
• Dirk Feldhusen, Matthias Heuft - Side Channel Analysis of Smart Cards (cache): these two implemented some nice ideas on how to attack an RSA private kay during processing on smart cards. Unfortunately, their english was quite weak, which made listening a major pain. And in the end they had to admit that their theoretical (and sucessfully simulated) attack didn't really work out in the real world.
• Daniel Bartlett - Automated Hacking via Google (cache): another potentially cool lecture that was torpedoed by an unexperienced speaker who seemed to be totally overwhelmed by the audience of about 900 people. His speech obviously was incomplete, and he failed to provide more than a few facts about attacks on php (and other) scripts, on how to find affected applications via google, and about automated attacks based on those results. He pointed out some possible improvements on the recent Santy worm, but didn't go into details either.
• Gerd Fittkau - Die Propagandaw�üste des realen Krieges (cache): one of the familiar, perfectly prepared and presented, anti-propaganda rants by Gerd Fittkau, the german-language answer to Michael Moore. Nice, but somehow I've heard similar talks before...
• Andy Müller Maguhn, Lars Weiler - CCC-Jahresrückblick (cache): the two CCC-speakers tell stories from their work over the last year. As usually, quite funny stuff
• Andreas Bogk, Felix von Leitner, FX of Penoelit, Michael Natterer - Das Literarische Code-Quartett (cache): four expierienced presenters show funny code snippets from different software, amongst them errors in the MySQL innodb log function and a buffer overflow in the mysql client libs.

Looked at some trouble on two of the newer 1&1 "rootserver XXL (cache)" boxen (3.06GHz P4 with Apollo Pro266 chipset) recently, which both seemed to have major disk I/O problems.

One of them, running a 2.4 series kernel, kept throwing messages like:

hda: dma_timer_expiry: dma status == 0x24
hda: DMA interrupt recovery
hda: lost interrupt

The other, with a recent 2.6 kernel, had no such warnings, but it's load went through the roof under heavy disk i/o, the system grew extremely sluggish, and sooner or later, the OOM killer would start to shoot out processes, although the memory usage was negligible, until the system died with a kernel panic. (Perhaps there is a problem with the OOM killer when a swap device is not available? Need to do some debugging here.)

Several runs at trying to optimize disk performace didn't produce any improvement, and no way of system tuning solved the problem.

Finally, on a whim, I tried booting the system with the "noapic" kernel option set, and suddenly all problems disappeared on both systems.

Seems something is wrong with the VIA Apollo IDE controller and Linux' interrupt routing via IO-APIC. Don't understand why I haven't seen anyone else having similar trouble, I assume there should be some more Linux setups on these 1&1 systems...

Just noticed that two of the webcomics sites from my list, Drow Tales (cache) and Krakow Studios (cache) have been defaced, and at least the first one obviously through a php script.

Don't know if a problem from the recent hardened-php advisory (cache) was involved, tough...

Update: Hello, Santy (cache). Seems I should have checked the ISC diary (cache) first...

I tried using the current portable OpenSSH (openssh-3.9p1) on an old OpenBSD 2.3 box.

Turns out it compiles fine as soon as it's persuaded not to use setreuid and the like, but authentication doesn't work - I assume that's because of the blowfish-encoded passwords.

So, in a fit of insanity, I started backporting the bsd_auth stuff from a current OpenBSD version, and actually got as far as an sshd compiled with BSD_AUTH starting up and telling me

Server listening on 0.0.0.0 port 2022.
sshd: invalid script: /usr/libexec/auth/login_passwd

when a user tries to log in.

After having a short look at the login_passwd sources, I decided to give up for now - seems the amount of dependency on things that simply don't exist in that old version is really too large.

I'm sticking with the old ssh.com ssh 1.2.33 for now...

(Yes, upgrading that installation to a current OpenBSD version would probably be the much better option, but some systems just don't die :) ...)

Just before my holidays (now 4 weeks ago, iiek), I discussed in this thread (cache) on the openbsd-misc mailinglist with someone who assumed OpenBSD had some kind of trouble with his client systems.

He was running a gateway, with pf enabled, that used ICMP redirects to point internal clients to a VPN router on his LAN. After the discussion went off-list, he explained his problems in more detail, and I came up with the following explanation:

 - you do filtering on the internal interface
 - the OpenBSD box gets the tcp syn, forwards the packet and sends the redirect
 - route is added on the client, tcp session is established, bypassing the OpenBSD gateway
 - after 10 minutes (or so) the route generated from the icmp redirect times out, and 
   packets are sent to the default gateway again
 - the OpenBSD box recieves packets from a tcp session it has no state for, and throws them away
 - your tcp session dies

So, if you're using icmp redirects, you can't do stateful filtering on the internal interface. 

Oops.

Now this one (cache) made me really laugh. ClamAV detecting the "viral" GPL license (cache) as virus would have made for an excellent April's Fools joke...

While I didn't gather enough information for a viable bug report yet, it seems I can't upgrade my old Samba 2 installations... (Which I would like to do, as communication between MacOS X clients and Samba 2 serves seems to be broken - uploads, especially, tend to time out or hang the finder on the client, while downloads are usually just abysmally slow.)

Unfortunately, Samba 3 makes problems with OS/2 clients (and I still have an OS/2 Warp 4 box here I use quite often) - obviously it now contains some sort of incomplete support for OS/2 extended attributes which can not be disabled in the server, and the OS/2 client will produce various access errors, depending on what it (or the user) is trying to do...

So, I'm keeping Samba 2 for Windows and OS/2 and NFS for the *NIX systems (including OSX) for now.

I really don't know what these guys think while they're developing their software...

No, I'm not talking about their latest buffer overflow (cache), but about the piece of crap that's called Windows XP ServicePack 2.

Again, Microsoft has managed to make work harder for admins while not helping everyone else too much. For example, after installation of SP2, the "trusted sites" pane of Internet Explorer has a new checkbox called something like "sites in this zone need a secure connection (https)" (translated back from the german version), which is activated by default.

If the user now tried to do the right thing, namely deactivating Active X and JScript for untrusted sites and adding *.windowsupdate.com and the like to the trusted sites list, where active content is then allowed, he won't be able to use windows update anymore, because that site for shure doesn't use https. The error messages say nothing about the changed behaviour.

Other example is the v5 windows update itself, which now needs the automatic update service and the BITS service active - but it isn't enough to start both before running windows update, it actually checks if those services are switched to automatic activation, and fails if not (another detail no error message mentions).

And I don't know whom they expect to keep the new nagging service (security center) activated for more than five minutes - there's no possibility for the user to tell the stupid thing that there is already a personal firewall active on the system (and a better one than the windows firewall, too) - it needs to be told by the installed firewall itself.

I pity everyone who needs to work with such a system.

Today, after posting to a mailing list:

Final-Recipient: rfc822; junk at localhost.nnn.nnn.com
Action: failed
Status: 5.0.0
Diagnostic-Code: X-Postfix; cannot access mailbox /var/mail/junk for user junk.
error writing message: File too large

Either, unsubscribing from an unwanted mailinglist is too complicated, and the mails are routed to a junk box instead, or they guy has a severly broken spam filter and never bothers to look at his filtered mails.

An article about Linux netfilter conntrack performance tweaking (cache)