Discussion:
Zero Length /sbin Files
(too old to reply)
N***@notreal.com
2018-05-07 17:19:21 UTC
Permalink
Raw Message
I am throwing this out for a possible answer but it is more out of
curiosity than a need to have an answer to solve a problem. As a result
you shouldn't waste your time unless you are curious as well.

Earlier today I had a need to run "lsmod" but when I did there was no
output at all, not even headings or error messages. Upon investigation
of found that /sbin/kmod had a file size of zero. Upon closer
examination I found a total of 14 files in /sbin with zero length as
follows.


-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/btrfs
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/debugreiserfs
-r-xr-xr-x 1 root root 0 Apr 11 18:16 /sbin/dmsetup
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/fatlabel
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/fsck.fat
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/hwclock
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/jfs_fsck
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/kmod
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/lnstat
-r-xr-xr-x 1 root root 0 Apr 11 18:16 /sbin/lvm
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/mkfs.fat
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/pccardctl
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/reiserfsck
-rwxr-xr-x 1 root root 0 Apr 11 18:16 /sbin/tune2fs

Those were all the files I could find with that exact date and time, but
I did find in the /tmp directory files with the date Apr 11 18:22 left
over from a clamav install of version 0.100.0.

The last paragraph could very well be a red-herring but if anyone has an
idea of what might have happened to result in those zero length files, I
would be interested in hearing it.

I have replaced the zero length files from an earlier backup and all
seems well so as I said, it is just idle curiosity that makes me wonder
what could have happened or if there is a common link between those
files.
Henrik Carlqvist
2018-05-08 19:02:05 UTC
Permalink
Raw Message
Post by N***@notreal.com
Earlier today I had a need to run "lsmod" but when I did there was no
output at all, not even headings or error messages. Upon investigation
of found that /sbin/kmod had a file size of zero. Upon closer
examination I found a total of 14 files in /sbin with zero length as
follows.
I can think of 3 different reasons for this to happen:

1) (most likely): Errors in the file system because of unclean shutdown.
I have seen this happen even on ext4 which is supposed to be a journaled
file system. Well at least the journaling seems to have prevented a
broken file system. In my case it was the patch binary which got
truncated to zero lenght after a power outage only some day after the
binary was updated by a slackware security update.

2) (less likely): Errors in the file system because of hardware errors.
Maybe you have a flaky disk with bad sectors, a glitchy SATA cable or a
flash memory which is about to wear out?

3) (even less likely): A malicious or clumsy process with root priveleges
messing with your files. All users are clumsy sometimes so running as
root should be avoided unless when necessary.

The annoying thing with those zero lenght executables is that they do not
do anything at all. They do not print any error message, they do not even
return any non zero return value. So for many cases it seems as if they
did what they were supposed to do when they in fact did nothing at all.

Example:

bash-4.1$ cp /dev/null /tmp/testexe
bash-4.1$ chmod u+x /tmp/testexe
bash-4.1$ ls -al /tmp/testexe
-rwxr--r-- 1 henca users 0 May 8 20:59 /tmp/testexe
bash-4.1$ /tmp/testexe
bash-4.1$ echo $?
0

regards Henrik
Ralph Spitzner
2018-05-09 08:34:44 UTC
Permalink
Raw Message
Henrik Carlqvist wrote:
[...]
Post by Henrik Carlqvist
1) (most likely): Errors in the file system because of unclean shutdown.
I have seen this happen even on ext4 which is supposed to be a journaled
file system. Well at least the journaling seems to have prevented a
broken file system. In my case it was the patch binary which got
truncated to zero lenght after a power outage only some day after the
binary was updated by a slackware security update.
I'll go with Henrik here, looks like the filesystem cache wasn't written.

If it where a security issue those are not the binaries I'd go for.

OTOH ***@notreal.com could do a find on suid root binaries, of course
he could only do that if he were real :-D .....

-rasp
Rob Windgassen
2018-05-09 17:48:29 UTC
Permalink
Raw Message
Post by Ralph Spitzner
[...]
Post by Henrik Carlqvist
1) (most likely): Errors in the file system because of unclean shutdown.
I have seen this happen even on ext4 which is supposed to be a journaled
file system. Well at least the journaling seems to have prevented a
broken file system. In my case it was the patch binary which got
truncated to zero lenght after a power outage only some day after the
binary was updated by a slackware security update.
I'll go with Henrik here, looks like the filesystem cache wasn't written.
If it where a security issue those are not the binaries I'd go for.
he could only do that if he were real :-D .....
He also may run fsck on the involved partition, may be more surprises
are lurking in that file system
--
Rob
Henrik Carlqvist
2018-05-10 10:01:51 UTC
Permalink
Raw Message
Post by Rob Windgassen
He also may run fsck on the involved partition, may be more surprises
are lurking in that file system
In the case of unclean shutdown that fsck should be done automagically by
the startup scripts at next reboot.

In the caes of broken hardware a manual fsck might be needed, but then I
would also look for signs of broken hardware in the output from dmesg and
try to fix or replace the broken hardware.

regards Henrik
N***@notreal.com
2018-05-10 20:50:39 UTC
Permalink
Raw Message
In article <pd15af$as8$***@dont-email.me>, ***@deadspam.com
says...
Post by Henrik Carlqvist
Post by Rob Windgassen
He also may run fsck on the involved partition, may be more surprises
are lurking in that file system
In the case of unclean shutdown that fsck should be done automagically by
the startup scripts at next reboot.
In the caes of broken hardware a manual fsck might be needed, but then I
would also look for signs of broken hardware in the output from dmesg and
try to fix or replace the broken hardware.
regards Henrik
Thank all of you for your comments. My first thought with momentary
panic was a compromised server but after checking for other damage and
various logs including an off site log of all logins I also came to the
conclusion that something else was amiss.

The most likely cause based on your possible scenarios, although I have
not experienced it before, is the unclean shutdown you suggested. My
town in NH was hit pretty hard by a line of thunderstorms late last week
and we actually lost all power for 22 hours due to numerous trees and
limbs on the power lines. During the initial phase the power went up and
down several times but when it seemed it was out for good I shut the
server down manually instead of waiting for the automatic shutdown from
the UPS. My thought was to save some battery life. I am guessing now
that I probably did not wait long enough before hitting the off switch
on the UPS. Now that this kind of thing is more than just theory to me,
I will try to be more cognizant of ensuring a clean shutdown. Thanks
again.

Loading...