Reading CSVs With Python's "csv" Module

Name: Reading CSVs With Python's "csv" Module
Uploaded: 2019-03-01T14:00:00+00:00
Duration: 4 min 13 s
Description: Now it’s time to start using Python to read CSV files. Here, I’ve got a simple CSV file that contains some employee data for two employees and has their name, department, and birthday month. Open up a new Python script and import csv. The first…

Reading and Writing CSV Files Joe Tatusko 04:13

In this video, you’ll learn how to read standard CSV files using Python’s built in csv module. There are two ways to read data from a CSV file using csv. The first method uses csv.Reader() and the second uses csv.DictReader().

csv.Reader() allows you to access CSV data using indexes and is ideal for simple CSV files. csv.DictReader() on the other hand is friendlier and easy to use, especially when working with large CSV files.

We’ll be using the following sample CSV file called employee_birthday.csv:

name,department,birthday month
John Smith,Accounting,November
Erica Meyers,IT,March

The following code samples show how to read CSV files using the two methods:

Using csv.Reader():

import csv

with open('employee_birthday.csv') as csv_file:
    csv_reader = csv.Reader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
        print(f'\t{row[0]} works in the {row[1]} department, and was born in {row[2]}')

Using csv.DictReader():

import csv

with open('employee_birthday.csv') as csv_file:
    csv_reader = csv.DictReader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
        print(f'\t{row["name"]} works in the {row["department"]} department, and was born in {row["month"]}')

00:00 Now it’s time to start using Python to read CSV files. Here, I’ve got a simple CSV file that contains some employee data for two employees and has their name, department, and birthday month. Open up a new Python script and import csv.

00:17 The first thing you need to do is actually open the CSV file.

00:27 Go ahead and call this csv_file. Next, create a csv_reader, which will just be csv.reader(). Pass in the csv_file and also a delimiter, which in this case, is just a simple comma (',').

00:45 Now you can create a line_counter, which we’ll just set equal to 0, and since the csv_reader generates a list of the different rows, you can go ahead and loop through for row in csv_reader:, and the first row contains all that header information, so when line_count == 0: you can print f'Column names are'—

01:16 and since that’s also a list of each item, just go ahead and .join() that row together. Increase the line_count, and handle everything else.

01:28 So, print(), and let’s start with a tab ('\t') here, and you can go ahead and just index off of that row, so the name will be the first item,

01:42 the department will be the second,

01:51 and the third item is the birthday month. Cool! Like before, increase the line_count and go ahead and print a summary, which will just be something like f'Processed {line_count} lines.' And that’s it! Save this, open up the terminal, and see what we get. All right, Column names are name, department, birthday month, and both employees’ information prints out here. And actually—ha ha—it looks like I printed the name twice.

02:30 This should be a 1 so that we access the second indexed item in there. Let’s try that again. There we go. Now we can see their department. Okay! So that’s pretty cool. You can open CSV files and parse them using Python.

02:47 Now, if your CSV files start getting larger, or if you have columns that you need to add or remove, the indexing can get very difficult, and as you can see, I made a typo here and printed the wrong thing.

02:58 Fortunately, csv has a different type of reader that allows you to use the header names. So go up here to the csv.reader, and you’re going to replace this with a DictReader, for dictionary.

03:10 Everything else will be the same. Go ahead and get rid of this else statement here and remove the indents on your every other row. And now, instead of picking apart the rows by their index, you can actually pass in the header name.

03:26 So what’s going on here is the DictReader (dictionary reader) looks at the first row and assumes that that’s the name of the row, so you can then refer to it by that name.

03:40 So if I go ahead, save this, we should be able to run it and get the same result. Yeah! So if you have that first row that contains the header information, you can use a different reader, and this is a little friendlier to use because you can actually see what you’re referring to. All right!

03:59 So now you can read standard CSV files. In the next video, we’re going to talk about changing some of the reader parameters to deal with those nonstandard cases. Thanks for watching, and I’ll see you there!

Walt Sorensen on March 31, 2021

Why use this CSV reader from import csv over Pandas read CSV?

import pandas as pd
file = csvfile.csv
df = pd.read_csv(file)

(especially when considering working with data in pandas?)

Bartosz Zaczyński RP Team on April 1, 2021

@Walt Sorensen Well, not everyone needs or knows about Pandas in the first place. It depends on your use case, but one advantage of the built-in csv module over Pandas is that it’s available in every Python distribution, while Pandas is not. That might be a problem if you don’t have Internet access or admin privileges, which is common in corporate environments.

naomifos on April 2, 2021

I have a probably weird question. Now I know that normally CSV files read through a row horizontally. But in my question problem I have, it is wanting me to read through it vertically so that my header is a key in a new dictionary and the values would be the things in the column vertically. How would I go about doing that?

Bartosz Zaczyński RP Team on April 2, 2021

@naomifos With the pandas library, that’s what happens implicitly because it stores data in columnar format:

>>> import pandas as pd
>>> df = pd.read_csv("data.csv")
>>> df["first_name"]
0     Joe
1    Anna
2     Bob
Name: first_name, dtype: object

>>> list(df["first_name"])
['Joe', 'Anna', 'Bob']

If you wanted to simulate the same behavior with pure Python, this is how you could go about it:

import csv
from collections import defaultdict, namedtuple

columns = defaultdict(list)
with open("data.csv", encoding="UTF-8") as file:
    reader = csv.reader(file)
    field_names = next(reader)
    Record = namedtuple("Record", field_names)
    for row in reader:
        record = Record(*row)
        for key, value in record._asdict().items():
            columns[key].append(value)

print(columns["first_name"])

Eduardo on May 11, 2021

I’m trying it with a csv uploaded on Github running my code in Google Colab.

url = 'https://raw.githubusercontent.com/UpInside/play_import_csv/master/municipios.csv'
with open(url) as csv_file:
  csv_reader = csv.reader(csv_file, delimiter = ',')
  line_count = 0
  for row in csv_reader:
    if line_count == 0:
      print(f'Column names are {", ".join(row)}')
      line_count += 1
    else:
      print(f'\t{row[0]} is the UF of {row[2]} city.')
      line_count += 1
  print(f'Processed {line_count} lines.')

But the return is the error:

FileNotFoundError: [Errno 2] No such file or directory: 'https://raw.githubusercontent.com/UpInside/play_import_csv/master/municipios.csv'

Leodanis Pozo Ramos RP Team on May 11, 2021

@Eduardo The problem here seems to be that you’re using open() to open an url instead of a local file. I think you should try downloading the file or using: docs.python.org/3/library/urllib.request.html#urllib.request.urlopen

Kim on Oct. 31, 2021

Hi, all.

I just went through this tutorial and with DictReader, it appears to have iterated rows through all of the rows, but only printed rows 0 and 2 and bypassed 1. Here is the snippet, which I believe is identical to the script in the tutorial.

with open(r'C:\Users\xxxx\xxxx\csv_example.txt') as csv_file:
    csv_reader = csv.DictReader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
        if line_count ==0:
            print(f'Column names are {", ".join(row)}')
            line_count += 1
        else:
            print(f'\t{row["name"]} works in the {row["department"]} department, and was born in {row["birthday month"]}.')
            line_count += 1
    print(f'Processed {line_count} lines.')

This code is similar to what I actually ran…I de-identified some details in my file location.

Any hints on the fix? Thanks!

Bartosz Zaczyński RP Team on Nov. 2, 2021

@Kim If you don’t explicitly provide a list of field names for your records, then DictReader will assume the first line in the CSV file is the header. It will try to get those field names from there, always skipping the first line during iteration:

name,department,birthday month
Joe Doe,IT,1999-01-01
Anna Smith,HR,2001-12-31
Bob Brown,Sales,2002-15-31

>>> import csv
>>> with open(r"example.csv") as csv_file:
...     csv_reader = csv.DictReader(csv_file, delimiter=",")
...     for row in csv_reader:
...         print(row)
...
{'name': 'Joe Doe', 'department': 'IT', 'birthday month': '1999-01-01'}
{'name': 'Anna Smith', 'department': 'HR', 'birthday month': '2001-12-31'}
{'name': 'Bob Brown', 'department': 'Sales', 'birthday month': '2002-15-31'}

Notice the loop starts at the second line in the file, which contains the actual data.

SirVeyor on Nov. 19, 2021

I learn just as much if not more from the comments. Thanks to all!

acamposla on Dec. 16, 2021

I am trying this approach

counter = 0
with open ("csv-sample-files/co-curricular-activities-ccas.csv") as singapore:
    singapore = csv.reader(singapore, delimiter = ",")
    line_count = 0
    for row in singapore:
        if line_count == 0:
            print(f"Las columnas son {', '.join(row)}")
            print(f"Hay un total de {len(row)} columnas")
            line_count +=1
        else:
            for i in range(5):
                print(f"\t{row[i]}")
                counter += 1
                if counter == 5:
                    print("---next line---")

I get this:

Las columnas son school_name, school_section, cca_grouping_desc, cca_generic_name, cca_customized_name
Hay un total de 5 columnas
    ADMIRALTY PRIMARY SCHOOL
    PRIMARY
    PHYSICAL SPORTS
    MODULAR CCA (SPORTS)
    SPORTS CLUB
---next line---
    ADMIRALTY PRIMARY SCHOOL
    PRIMARY
    VISUAL AND PERFORMING ARTS
    ART AND CRAFTS
    VISUAL ARTS CLUB
    ADMIRALTY PRIMARY SCHOOL
    PRIMARY
    CLUBS AND SOCIETIES
    ENGLISH LANGUAGE, DRAMA AND DEBATING
    ENGLISH LANGUAGE AND DRAMA
    ADMIRALTY PRIMARY SCHOOL

However i would like to print “Next line” every 5 lines (columns)

Become a Member to join the conversation.