Linux oh-shit moments

The hackability of Linux can sometimes lead to really bad screw-ups. Let's trade some stories.

I just had my worst Linux screw-up, trying to install Steam Link on an ancient Raspberry Pi. The thing was neglected for so long that when I started with an "apt-get upgrade", the downloaded packages are no longer compatible with the current dpkg. And apt is confused into an unusable state.

As a hacker and not wanting to download a new Raspbian, I came up with a hacky solution: update dpkg manually, then use dpkg to install steamlink.

So I googled for "how to extract deb", used ar and tar to extract the (fortunately) downloaded packages in /var/cache to /tmp. And I cp -r the stuff to /.

And things got worse. The old dpkg was just incompatible. The new dpkg won't even load: it depends on a newer libc. No problem, I thought. And I proceeded to ar / tar the libc package into /tmp and cp -r them to / as well.

And cp segfaulted. Now things got really interesting. Previously it was just dpkg, now ar / tar stopped working as well! I can't even ls /lib! Fortunately, I vaguely remember reading a story online about a similar situation. I did lose libc. But I still have one in-memory copy in my running bash session. So there should still be a way out…

Being a developer, I managed to recall LD_LIBRARY_PATH. And at this point I finally remembered we had symlinks in /lib and I should probably cp -ar.

One LD_LIBRARY_PATH=/tmp/... cp -ar later, I regained a working libc and access to basic unix commands. A few more ar ; tar ; cp sessions later, I got all the dependencies of dpkg covered.

Only to find that since I broke dpkg during apt-get upgrade, I never got to download steamlink itself. So I need to fix apt as well. Which involved dozens of dpkg -i.

And finally, after hours of hacking and a final working apt install steamlink, I discovered the icing on the cake: turns out my Pi was a Raspberry Pi 2 thus incompatible with steamlink.