Add Celery to Your Django Project

00:00 Add celery to your Django project. The final puzzle piece is connecting the Django app as a message producer to the task queue. Navigate into the Django celery management app folder and create a new file called celery.py.

00:16 Celery recommends using this module to define the celery application instance. Open the file in your text editor or IDE and add the code seen on screen.

00:26 You only need a few lines of code to get up and running. First, you import the built-in os module, which you may well be familiar with. You’ll use it later to set an environment variable.

00:37 Next, you import celery from the celery package. This will be used to create the celery application instance. Here you use os.environ.setdefault to ensure that your Django project’s settings.py module is accessible through the DJANGO_SETTINGS_MODULE key.

00:56 Next, you create the celery application instance and provide the name of the main module as an argument in the context of your Django application.

01:04 The main module is the Django app that contains celery.py, so you pass django_celery. In this line, you define the Django settings file as the configuration file for celery and provide a namespace CELERY.

01:20 You’ll need to prepend the namespace value followed by an underscore to every configuration variable related to celery. You could define a different settings file, but keeping the celery configuration in Django settings file allows you to stick with a single central place for configuration.

01:38 With this final line, you tell your celery application instance to automatically find all tasks in each app of your Django project. This works as long as you stick to the structure of reusable apps and define all celery tasks.

01:52 For an app in a dedicated tasks.py module, you’ll create and populate this file for your django_celery app when you refactor the email sending code later on in the course.

02:06 With celery.py set up and attempting to fetch the necessary celery settings from your settings.py file, next head over there and add these settings entries to the bottom of the file.

02:29 These two entries give your celery application instance enough information to know where to send messages and where to record the results. Because you’re using Redis as both your message broker and your database backend.

02:42 Both URLs point to the same address. This final line will remove a deprecation warning about a future change in celery behavior, making it clearer to see what’s going on when you run it.

02:55 Note that the two configured URLs could also point to different servers and services. For example, you could make the changes seen on screen to use RabbitMQ as your message broker and Redis as the results backend.

03:10 When you run your app in production, you’d replace these URLs with the production locations of each service.

03:17 Note, the CELERY_ namespace at the beginning of these settings variables. You need to add this because of the namespace='CELERY' argument that you passed to app.config_from_object in celery.py earlier on.

03:33 At this point, you are nearly done integrating celery into the web app. The final edition goes into __init__.py of the management app.

03:42 Open the file in your text editor in a default Django project. Each app folder has a __init__.py file, which helps mark it as a module. The file is empty by default, but you can add code to influence the import behavior.

03:59 First, app is imported as celery_app, and then to make sure that your celery app is loaded when you start Django, you should add it to __all__.

04:09 Loading the celery app on Django startup ensures that the @shared_task decorator will use it correctly.

04:18 You’ll learn more about that in the next section.

04:23 It’s time to test your setup. Remember that the process you’re setting up requires at least three services to run at the same time. The producer, which is your Django app, the message broker, which is the Redis server, and the consumer, which is your celery app.

04:40 Because you are using Redis, you’ll get the database backend as a bonus without running another service.

04:48 Open up three separate terminal windows and start the programs. If they’re not already running. Serve the web app with Django’s development server in the first window.

05:04 Then start the Redis server in the second terminal window. In case you stopped it earlier,

05:13 the Redis server command is the only one of the three that you can run outside your virtual environment, but you will need to make sure that it’s active in the other two terminal windows.

05:25 If you’ve already got Redis server running in the background and another window, you’ll see an error message similar to the one that you can see on screen.

05:34 Finally, you can now also start celery correctly without running into an error message.

05:43 When starting celery with this command, you provide the name of the module that contains your celery app instance, django_celery to the -A option, while the error message that you saw when running celery before you installed Redis is gone.

05:58 You may still see a warning related to Django’s DEBUG setting. You can ignore this warning for the example app, but you should always set DEBUG to False.

06:06 Before deploying a site into production,

06:11 you only need to add codes to the three mentioned files to integrate celery into your Django app and prepare it for processing asynchronous tasks.

06:20 With this base setup complete, you are ready to write a task that you can hand off to celery. So in the next section of the course, you’ll refactor send email to call an asynchronous celery task instead of handling the email synchronously in Django.