Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to hundreds of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please see our video player troubleshooting guide to resolve the issue.

Creating Data Pipelines With Generators

In this lesson, you’ll learn how to use generator expressions to build a data pipeline. Data pipelines allow you to string together code to process large datasets or streams of data without maxing out your machine’s memory.

For this example, you’ll use a CSV file that is pulled from the TechCrunch Continental USA dataset, which describes funding rounds and dollar amounts for various startups based in the USA. Click the link under Supporting Material to download the dataset included with the sample code for this course.

Comments & Discussion

nelsonblue24 on June 28, 2020

If I understand correctly: * Lines 6-10 contain still another generator. * Line 11 sums all the values generated by that generator.

If I am correct, I think it would help to state those facts explicitly in the video.

Anonymous on July 8, 2020

I computed the sum for the series A funding using my own code, and keep getting $4380015000 instead of $4376015000 (using the code in the video). Can’t figure out why my sum is larger. I used csv.DictReader, Pandas DataFrame and also Excel .

Thanks!

Become a Member to join the conversation.