0

In Linux, "everything is a file". This makes it so that malicious code can practically be put into any file.

If I create a malicious .pdf file and upload it to github, this file can be exploited when processed in a program, right?

What about a .MD file where there is only text? I guess that it can also be exploited if there is a vulnerability in the reader program e.g Obsidian?

Does git store the actual bytes of a file, or only the readable text in certain cases?

E.g a abc.txt file containing "abc", can I clone that abc.txt file safely even though some bytes could have been modified?

Is there a difference between copying the raw page from github and cloning a file?

1
  • You have edited this question such that the answer with votes no longer applies. I would post a new question if you want to ask something different.
    – schroeder
    Commented Jul 8 at 22:27

1 Answer 1

5

In Linux, "everything is a file". This makes it so that malicious code can practically be put into any file.

That's, like, not even wrong? You've just fundamentally misunderstood what that saying means, what files are, or both. What the saying means is "every kind of user-accessible operating system object is in the file system somewhere and can be interacted with using file APIs". Devices, processes, environment variables, pipes, sockets, even file descriptors. Note that none of those are "files" in the sense of "a fixed-length series of bytes that could be stored in a block device" a.k.a. "normal files". Similarly, for many of those things at least some of the file APIs will behave "strangely" or do nothing at all; you can't seek on a printer device or get the length of a pipe or so on, and while you can both read and write a socket, you will not by default be able to read back what you wrote before, etc.

With that said, yes, malicious code can in theory be put into, at least, any "file" that implements a write operation, or returned from any that implement read. Most types of objects you do this to won't be executable, but they could certainly exploit a vulnerability in a parser that reads the data and tries to process it, or that accepts written data to a "file" node and processes that. Of course, that's exactly the same on Windows and most other operating systems; very few treat "things that can have code in them" or "things that can implement read and/or write" as a special class of object; you can in fact use file APIs on a number of object types in Windows that are not in the user-exposed file system at all.

Does git store the actual bytes of a file, or only the readable text in certain cases?

Obviously it's either actual bytes or a lossless encoding of them (such as a lossless compression), otherwise you wouldn't be able to download the unmodified file again. Git clients can (and sometimes do) modify a file before committing or after fetching (or maybe just after checkout?) - most commonly to normalize line endings - but that's optional and github isn't usually doing that modification itself anyhow. Of course, "the actual bytes of a file" is itself somewhat nebulous; if I read a file using file system APIs I will get a consistent view of it, but if I look at it on disk it might be fragmented (non-contiguous in storage), sparse (contain regions not backed by storage at all and assumed to be null), compressed (losslessly, obviously), and encrypted (with transparent encryption on write / decryption on read by a storage driver). And that's just a "normal file", and doesn't even get into things like extended attributes or alternate data streams or any of the other things that some file systems implement. And of course when you copy or move a file between file systems - much less between machines - all of that can change.

In any case, it's certainly not going to be "only the readable text" except in the case of plain-text files with no byte-order marks or similar non-printing characters. The line endings thing is as close as it gets.

Is there a difference between copying the raw page from github and cloning a file?

Shouldn't be, other than that cloning the repo gets you the other files and the history of each file and various other metadata. Strictly speaking, Github can do whatever it wants with the "raw file" view, but it would be really weird if what it was doing was anything other than "send the HTTP metadata and then pipe an input stream into the HTTP socket". Of course, that input stream isn't necessarily a file input stream; it could be a database object (unlikely given the sizes in question, but not conceptually impossible), an HTTP or other network protocol connection to a remote storage service like AWS S3/Azure Blob Storage, a decompression stream that is running a codec on some actual storage stream, or so on.


Anyhow, to answer the question in the title, nobody outside of Github (aside from the people Github has let look at their systems for other reasons, like third-party penetration tests) actually knows. Github's own source code and infrastructure are private; the closest we've seen to it is an illegal leak of the Github Enterprise source (which probably shares a lot with actual github.com source but at minimum is designed to run on corporate infrastructure that likely differs from what Github itself uses). However, you can rest assured that Github is storing some representation of the full contents of files that get uploaded, even if that representation is not necessarily what you think of as a "file" and may very well not be a byte-for-byte match at rest to what you get when you request the file from Github. After all, it's entirely possible that what you see in an IDE or even a hex editor viewing a file isn't a byte-for-byte match to how the data is stored on your own machine, either.

With all that said, from a security standpoint, treat files from Github the same as any other files downloaded off the Internet. Definitely don't just blindly trust that they aren't malicious - Github allows hosting malware and exploit code, so long as it's clearly labeled as such, but also people's projects sometimes unintentionally or intentionally-but-secretly contain malicious files too - and remember that you need to consider how trustworthy both Github and the person who uploaded the file to Github might be. Additionally, note that git itself has had vulnerabilities at times, such that cloning a malicious repository could overwrite files it wasn't supposed to or other things like that. After all, git is a parser too. Still, absent such cases, merely cloning a repository or viewing a raw file (which should be fine unless your browser has a vulnerability in handling that file type) is generally safe; in most cases you have to do something with a file (execute or otherwise "open" it in a way that involves in-depth parsing) beyond simple viewing for that file to do anything malicious to your system.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .