Reading Multiple File Inputs
In this lesson, I’ll show you how to read from multiple file inputs concurrently with the
fileinput module, which is a cool little trick that you can use to essentially make reading multiple file inputs into like reading just one file input.
So, the arguments provided to the execution of this Python function—Python module, I should say, so
sys.argv. And it takes all of the filenames from the
filenames list, and then returns, essentially, just one input stream that you can then just iterate through like any other input stream.
00:45 It also provides some useful information about each line in the output, like the line number, whether the line is the first line in its file, and so on. It gives you some stuff to work with instead of just pretending that it’s all one big file, so that’s pretty helpful.
What I want to do is go through a quick little project with you to show you the importance, or at least the usefulness, of
fileinput. What I’m going to do—first, after listing and showing you that I have just an empty directory—my goal is to create a crude version of
cat, which is a Linux utility that reads multiple files and displays their contents
stdout (standard out). You can see how this would be useful because you could combine this with
os.listdir() to get a nice, concise summary of all of the contents of the things in your list.
01:48 Well, it may not be concise, if your files aren’t concise. But what I mean is just that you could look over the contents of all the files in a directory really easily, if you could have this utility.
This is something that’s very common to use in Linux and in the macOS terminal, and so on. It’s a super-useful utility. So, the
fileinput module can be used to do that, but first I’m going to need to create some files.
So I’ll say this, I’ll say,
flist = ["file1.txt", "file2.txt", "file3.txt"], and that should be enough for our purposes. And now I’ll say,
for fname in flist:, I want to say
with open(fname) in write mode
as f:, I will just go and say,
f.write—and I’ll make it an f-string—there’s a lot of f’s going on here, but that’s how it goes sometimes—
f"Hi I'm file #"—oh, and I want to make sure I keep track of the file number, so I’ll use the enumerate pattern here, and I’ll say
enumerate(flist)—enumerate function, I should say.
They’re in kind of an odd order, but don’t worry about that. That’s just how it goes with your system sometimes. Now, I have some files to test this utility on, and I want to actually say here,
It will take in a file list, and then it will say
lines = fileinput.input() and I have to call it on the file list. And if I don’t put any files in here, then it will actually just use the system arguments.
03:39 So when I call this Python file that I would be actually creating here from the command line, it would use those arguments. But as of now, I’m going to make sure that it uses this file list that’s passed in.
04:21 So, that’ll work well, it will print out that we’re processing a file. And maybe one more thing that I want to do is just give that—I’m just going to give it a newline so that it’s clear, or at least that it’s obvious when you look through this output, that there’s a new file being processed.
So, now what I can do is I can just print out the line and maybe I’ll have a little thing here, like an arrow that just says that this is a line. So, this is really pretty much all you need for a
So if I say now, I’ll just try calling
flist, luckily I already have that defined. What it does is it says
Processing file: file1.txt. -> Hi I'm file #1,
Processing file: file2.txt. -> Hi I'm file #2,
Processing file: file3.txt. -> Hi I'm file #3.
05:19 And luckily this does make it a little bit easier. It processes the file, and then it has the lines, processes the file, has the lines. So, that was some awkward spacing, but with an extra newline, it works out just great.
So now, you have a basically functional version of the Linux utility
cat, all made with
fileinput. And it has several more useful functions in it than just
filename(), so I encourage you to take a look at those and see how you can incorporate that in your own programming because it can be really useful to have a bunch of files that you want to process all in the same way.
You can just process them all at once with
fileinput.input(), which is quite convenient. It’s especially convenient if you want to make command-line applications because it defaults to the
Become a Member to join the conversation.