Writing CSVs With Python's "csv" Module
In this lesson, you’ll learn how to write data to CSV files using the two csv
writer methods csv.writer
and csv.DictWriter
. The lesson also covers how to quote specific fields using constants such as csv.QUOTE_MINIMAL
, csv.QUOTE_NONNUMERIC
, csv.QUOTE_NONE
, and csv.QUOTE_ALL
.
00:00
Now it’s time to start writing CSV files. Just like reading, you can write CSV files line by line, or by using a dictionary. Open up a new Python script, and let’s get started by importing csv
.
00:14
Also like before, you’ll need to use a with open()
statement to open that file, which you can call something like 'employee_file.csv'
.
00:25
Make sure you set the mode
equal to write, and just call that something like employee_file
. Next, create an employee_writer
object, which you can set equal to a csv.writer()
, pass in the employee_file
, set the delimiter
equal to a standard comma, a quotechar
,
00:50
which can just be a single double quotes. And unlike reading the file, you’ll need to set this parameter called quoting
, which—for now—just put QUOTE_MINIMAL
.
01:04
I’ll move this out of the way so that we can see. And now with that employee_writer
, you can just call something like .writerow()
and then pass in a list of the items that you’d like to add.
01:18
So, 'John Smith'
from 'Accounting'
,
01:24
and he was born in 'November'
. And then employee_writer.writerow()
, and 'Erica Meyers'
from 'IT'
who was born in 'March'
.
01:41
Okay. Save this, open up a terminal, and run it. All right. Didn’t see any errors. Let’s go to this directory. And here you can see a CSV was created using the delimiter
you specified and the quoting
, which in this case was minimal—so, QUOTE_MINIMAL
just means that it’ll add quotes if the field contains the delimiter
or the quotechar
.
02:09
Let’s take a look at some of the other options. You can also do QUOTE_ALL
instead of QUOTE_MINIMAL
, which will add quotes to everything, no matter what.
02:18
So, let’s actually save this as a new file, run that, and here you can see every field now is wrapped in those quotes. The third option is to QUOTE_NONNUMERIC
, which—just like it sounds—will go and quote any fields that don’t contain numeric data.
02:38
And then finally, you can use something called QUOTE_NONE
, which won’t add any quotes and will just escape any characters that it needs to. And if you’re going to use this, you need to add in an escapechar
, which we’ll set equal—just like we had before—to the pipe.
02:54 So, let’s save this as a new file, and just to make sure it works, let’s put some commas into the names here. Go ahead, run that, open up the new file, and you can see that there are no quotes and it used the escape character before the commas that we added to the data fields.
03:13
So, let’s go close this terminal out, go back to this, and let’s see how to write a CSV file using a dictionary. I’m just going to change the filename here to something like that, and now, you need to add in a fieldnames
variable, so this’ll just be something like ['name', 'department']
—actually, just to keep things short—and 'birth_month'
. And kind of like before, instead of just the regular writer
, this will now be a DictWriter
(dictionary writer).
03:50
You can get rid of all of this information here and just pass in fieldnames=fieldnames
—which I can tell from my autofill, I forgot an s
up here, so let’s fix that right now.
04:04
And what’s a little different now is before you actually start writing, you need to call the .writeheader()
method, and that’ll add that first row that’ll contain the header information.
04:16
And for these, instead of passing in lists, they’ll now be dictionaries, so 'name'
equals 'John Smith'
, 'dept'
is 'Accounting'
, and 'birth_month'
is 'November'
.
04:41 Let’s just go ahead and copy this row,
04:46
and we’ll fill out Erica’s data down here. So, 'Erica Meyers'
,
04:55
'IT'
, and 'March'
. Alrighty! Let’s try to run this.
05:03
Cool! Open up the file, and now you can see that we have the same information down here, but we also have that header information up top. And that’s it! Now you can write CSV files using the csv
library in Python!
05:16
The last thing we’ll cover is reading and writing CSV files, but using the pandas
library instead. You could have entire courses on pandas
, but we’re just going to focus on using it to get CSVs into it and out of it.
Joe Tatusko RP Team on July 5, 2019
Hi Robert! I’m using Atom and platformio-ide-terminal for the integrated terminal at the bottom in these videos.
Robert on July 9, 2019
Thank you, I’ll check those out
Dave Wilson on Feb. 23, 2020
Such a great tutorial, Joe - as always!
I’m not sure why, but my csv’s printed rows on every other line. I was able to fix it by adding a lineterminator to the employee_writer definition:
employee_writer = csv.DictWriter(employee_file, fieldnames=fieldnames, lineterminator = '\n')
hayatmor on March 15, 2020
In python3 in order to work without double space you shall add:
with open('employee_file.csv', mode='w', newline='') as employee_file:
sweir12525 on Sept. 24, 2020
Hi, I got a reply to a prior question regarding why I am getting a syntax error when I put in pip install pandas. The RP Team member, Bartosz, answered telling me what a csv file was and that any text editor can open them. Problem is that I know exactly what a CSV file is, I am using PYcharm and Thorny and on both applications, when I enter the script in this lesson, I get a syntax error. I have installed Anaconda and when I put in the entire script I get attribution error. When I debug, it says I should try calling something other than pandas. So, please explain to me what I am doing wrong since all I am doing is following the lesson plan.
Ricky White RP Team on Sept. 25, 2020
Hi @sweir12525. It might be useful in this case if you copy the bottom part of the Traceback error. Just above where it says Syntax error, it’ll show you which line of code is causing the problem. It would be helpful if we could see that so we can help you further.
sweir12525 on Sept. 29, 2020
Thanks, Ricky. I did manage to finally figure this out after a ton of reading in Anaconda.
karmajna on Sept. 29, 2020
Added newline
variable to open function to not print on every other line as other mentioned.
with open('employee_file.csv', mode='w', newline='') as employee_file:
Kim on Nov. 4, 2021
Hi.
I received an error when trying to write a csv file with dictionary. Here is my code
with open("C:/Users/xxxx/Desktop/dictionary_file.txt",'w')as employee_file:
fieldnames=['name','dept','birth_month']
employee_writer=csv.DictWriter(employee_file, fieldnames=fieldnames)
employee_writer.writeheader()
employee_writer.writerow([{'name':'John Smith','dept':'Accounting','birth_month': 'November'}])
employee_writer.writerow([{'name':'Erica Meyers','dept': 'IT','birth_month': 'March'}])
Here is the error I received:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-26-bb18ce957124> in <module>
4
5 employee_writer.writeheader()
----> 6 employee_writer.writerow([{'name':'John Smith','dept':'Accounting','birth_month': 'November'}])
7 employee_writer.writerow([{'name':'Erica Meyers','dept': 'IT','birth_month': 'March'}])
~\anaconda3\envs\R-base\lib\csv.py in writerow(self, rowdict)
152
153 def writerow(self, rowdict):
--> 154 return self.writer.writerow(self._dict_to_list(rowdict))
155
156 def writerows(self, rowdicts):
~\anaconda3\envs\R-base\lib\csv.py in _dict_to_list(self, rowdict)
145 def _dict_to_list(self, rowdict):
146 if self.extrasaction == "raise":
--> 147 wrong_fields = rowdict.keys() - self.fieldnames
148 if wrong_fields:
149 raise ValueError("dict contains fields not in fieldnames: "
AttributeError: 'list' object has no attribute 'keys'.
What do I need to do to fix? Thanks!
Bartosz Zaczyński RP Team on Nov. 5, 2021
@Kim Unlike the regular CSV writer, the dict writer’s .writerow()
method expects a dictionary and not a list:
import csv
with open("employees.csv", "w") as file:
writer = csv.DictWriter(file, fieldnames=["name", "dept", "birth_month"])
writer.writeheader()
writer.writerow(
{
"name": "John Smith",
"dept": "Accounting",
"birth_month": "November",
}
)
writer.writerow(
{
"name": "Erica Meyers",
"dept": "IT",
"birth_month": "March"
}
)
Kim on Nov. 7, 2021
OK. Thank you! I’ll use what you have above.
Realblogger on Sept. 13, 2023
Hello, I receive error : AttributeError: ‘module’ object has no attribute ‘writer’
Any suggestions.
Thanks,
walterrieppi on Feb. 5, 2024
When i try to repeat the example where i write a csv file with the quoting parameter set to QUOTE_ALL i have this situation
employee_writer.writerow([‘John Smith’,’Accounting’,’November’]) employee_writer.writerow([‘Erica Meyers’,’IT’,’March’])
In my new file created by the program i have these row
John Smith,”Accounting”,”November” Erica Meyers,”IT”,”March”
Why the first element is written without the ” ” ????? thanks in advance to all
Become a Member to join the conversation.
Robert on July 1, 2019
Hello,
What is the IDE or editor that you are using?
Robert