Efficient String Concatenation in Python

Efficient String Concatenation in Python

by Leodanis Pozo Ramos May 03, 2023 basics best-practices python

String concatenation is a common operation in programming. It involves joining two or more strings to create a single new string. You’ll find several tools and techniques for concatenating strings in Python, each with its own pros and cons.

In this tutorial, you’ll go through the most common tools and techniques for concatenating strings. You’ll also code examples for each of them, which will help you choose the best option for your specific problems.

In this tutorial, you’ll:

  • Understand what string concatenation is and why it’s useful
  • Concatenate two strings using the concatenation operators, + and +=
  • Efficiently concatenate multiple strings using the .join() method
  • Explore alternative concatenation techniques like string literals, StringIO, and print()

To get the most out of this tutorial, you should have a basic understanding of Python, especially its built-in string data type.

Doing String Concatenation With Python’s Plus Operator (+)

String concatenation is a pretty common operation consisting of joining two or more strings together end to end to build a final string. Perhaps the quickest way to achieve concatenation is to take two separate strings and combine them with the plus operator (+), which is known as the concatenation operator in this context:

Python
>>> "Hello, " + "Pythonista!"
'Hello, Pythonista!'

>>> head = "String Concatenation "
>>> tail = "is Fun in Python!"
>>> head + tail
'String Concatenation is Fun in Python!'

Using the concatenation operator to join two strings provides a quick solution for concatenating only a few strings.

For a more realistic example, say you have an output line that will print an informative message based on specific criteria. The beginning of the message might always be the same. However, the end of the message will vary depending on different criteria. In this situation, you can take advantage of the concatenation operator:

Python
>>> def age_group(age):
...     if 0 <= age <= 9:
...         result = "a Child!"
...     elif 9 < age <= 18:
...         result = "an Adolescent!"
...     elif 19 < age <= 65:
...         result = "an Adult!"
...     else:
...         result = "in your Golden Years!"
...     print("You are " + result)
...

>>> age_group(29)
You are an Adult!
>>> age_group(14)
You are an Adolescent!
>>> age_group(68)
You are in your Golden Years!

In the above example, age_group() prints a final message constructed with a common prefix and the string resulting from the conditional statement. In this type of use case, the plus operator is your best option for quick string concatenation in Python.

The concatenation operator has an augmented version that provides a shortcut for concatenating two strings together. The augmented concatenation operator (+=) has the following syntax:

Python
string += other_string

This expression will concatenate the content of string with the content of other_string. It’s equivalent to saying string = string + other_string.

Here’s a short example of how the augmented concatenation operator works in practice:

Python
>>> word = "Py"
>>> word += "tho"
>>> word += "nis"
>>> word += "ta"
>>> word
'Pythonista'

In this example, every augmented assignment adds a new syllable to the final word using the += operator. This concatenation technique can be useful when you have several strings in a list or any other iterable and want to concatenate them in a for loop:

Python
>>> def concatenate(iterable, sep=" "):
...     sentence = iterable[0]
...     for word in iterable[1:]:
...         sentence += (sep + word)
...     return sentence
...

>>> concatenate(["Hello,", "World!", "I", "am", "a", "Pythonista!"])
'Hello, World! I am a Pythonista!'

Inside the loop, you use the augmented concatenation operator to quickly concatenate several strings in a loop. Later you’ll learn about .join(), which is an even better way to concatenate a list of strings.

Python’s concatenation operators can only concatenate string objects. If you use them with a different data type, then you get a TypeError:

Python
>>> "The result is: " + 42
Traceback (most recent call last):
    ...
TypeError: can only concatenate str (not "int") to str

>>> "Your favorite fruits are: " + ["apple", "grape"]
Traceback (most recent call last):
    ...
TypeError: can only concatenate str (not "list") to str

The concatenation operators don’t accept operands of different types. They only concatenate strings. A work-around to this issue is to explicitly use the built-in str() function to convert the target object into its string representation before running the actual concatenation:

Python
>>> "The result is: " + str(42)
'The result is: 42'

By calling str() with your integer number as an argument, you’re retrieving the string representation of 42, which you can then concatenate to the initial string because both are now string objects.

String concatenation using + and its augmented variation, +=, can be handy when you only need to concatenate a few strings. However, these operators aren’t an efficient choice for joining many strings into a single one. Why? Python strings are immutable, so you can’t change their value in place. Therefore, every time you use a concatenation operator, you’re creating a new string object.

This behavior implies extra memory consumption and processing time because creating a new string uses both resources. So, the concatenation will be costly in two dimensions: memory consumption and execution time. Fortunately, Python has an efficient tool for you to deal with concatenating multiple strings. That tool is the .join() method from the str class.

Efficiently Concatenating Many Strings With .join() in Python

You can call the .join() method on a string object that will work as a separator in the string concatenation process. Given an iterable of strings, .join() efficiently concatenates all the contained strings together into one:

Python
>>> " ".join(["Hello,", "World!", "I", "am", "a", "Pythonista!"])
'Hello, World! I am a Pythonista!'

In this example, you call .join() on a whitespace character, which is a concrete object of the built-in str class expressed as a literal. This character is inserted between the strings in the input list, generating a single string.

The .join() method is cleaner, more Pythonic, and more readable than concatenating many strings together in a loop using the augmented concatenation operator (+=), as you saw before. An explicit loop is way more complex and harder to understand than the equivalent call to .join(). The .join() method is also faster and more efficient regarding memory usage.

Unlike the concatenation operators, Python’s .join() doesn’t create new intermediate strings in each iteration. Instead, it creates a single new string object by joining the elements from the input iterable with the selected separator string. This behavior is more efficient than using the regular concatenation operators in a loop.

It’s important to note that .join() doesn’t allow you to concatenate non-string objects directly:

Python
>>> "; ".join([1, 2, 3, 4, 5])
Traceback (most recent call last):
    ...
TypeError: sequence item 0: expected str instance, int found

When you try to join non-string objects using .join(), you get a TypeError, which is consistent with the behavior of concatenation operators. Again, to work around this behavior, you can take advantage of str() and a generator expression:

Python
>>> numbers = [1, 2, 3, 4, 5]

>>> "; ".join(str(number) for number in numbers)
'1; 2; 3; 4; 5'

The generator expression in the call to .join() converts every number into a string object before running the concatenation and producing the final string. With this technique, you can concatenate objects of different types into a string.

Doing Repeated Concatenation With the Star Operator (*)

You can also use the star operator (*) to concatenate strings in Python. In this context, this operator is known as the repeated concatenation operator. It works by repeating a string a certain number of times. Its syntax is shown below, along with its augmented variation:

Python
string = string * n

string *= n

The repetition operator takes two operands. The first operand is the string that you want to repeat in the concatenation, while the second operand is an integer number representing how many times you want to repeat the target string. The augmented syntax is equivalent to the regular one but shorter.

A common example of using this concatenation tool is when you need to generate a separator string to use in tabular outputs. For example, say you’ve read a CSV file with information about people into a list of lists:

Python
>>> data = [
...    ["Name", "Age", "Hometown"],
...    ["Alice", "25", "New York"],
...    ["Bob", "30", "Los Angeles"],
...    ["Charlie", "35", "Chicago"]
... ]

The first row of your list contains the table headers. Now you want to display this info in a table. You can do something like the following:

Python
>>> def display_table(data):
...     max_len = max(len(header) for header in data[0])
...     sep = "-" * max_len
...     for row in data:
...         print("|".join(header.ljust(max_len) for header in row))
...         if row == data[0]:
...             print("|".join(sep for _ in row))
...

>>> display_table(data)
Name    |Age     |Hometown
--------|--------|--------
Alice   |25      |New York
Bob     |30      |Los Angeles
Charlie |35      |Chicago

In this example, you use the * operator to repeat the string "-" as many times as defined in max_len, which holds the number of characters in the longest table header. The loop prints the data in a tabular format. Note how the conditional statement prints a separation line between the headers and the actual data.

Exploring Other Tools for String Concatenation

Python is a highly flexible and versatile programming language. Even though the Zen of Python states that there should be one—and preferably only one—obvious way to do it, you’ll often find several options for running a given computation or performing a certain action in Python. String concatenation is no exception to this behavior.

In the following sections, you’ll learn about a few additional techniques and tools that you can use for string concatenation.

Taking Advantage of String Literal Concatenation

Another quick way to concatenate multiple strings in Python is to write the string literals consecutively:

Python
>>> "Hello," " " "World!"
'Hello, World!'

>>> message = "To: " "The Python Community" " -> " "Welcome Folks!"
>>> message
'To: The Python Community -> Welcome Folks!'

In these examples, you can see that Python automatically merges multiple strings into a single one when you place them side by side. This feature is known as string literal concatenation, and it’s documented as an intentional behavior of Python. Note that you can even define variables using this feature.

Some common use cases for string literal concatenation include:

  • Using different quoting styles without having to escape quote symbols
  • Splitting long strings conveniently across multiple lines
  • Adding comments to parts of strings

For example, say that you need to use double quotes and apostrophes in the same string:

Python
>>> phrase = (
...     "Guido's talk wasn't named "  # String with apostrophes
...     '"Feeding CPython to ChatGPT"'  # String with double quotes
... )

>>> print(phrase)
Guido's talk wasn't named "Feeding CPython to ChatGPT"

In this example, you take advantage of string literal concatenation to add comments to different parts of your string and to escape double and single quotes within your final string.

This feature may seem neat at first glance. However, it can be a good way to shoot yourself in the foot. For example, say that you’re writing a list of strings and accidentally forget an intermediate comma:

Python
>>> hobbies = [
...     "Reading",
...     "Writing",
...     "Painting",
...     "Drawing",
...     "Sculpting" # Accidentally missing comma
...     "Gardening",
...     "Cooking",
...     "Baking",
... ]

>>> hobbies
[
    'Reading',
    'Writing',
    'Painting',
    'Drawing',
    'SculptingGardening',
    'Cooking',
    'Baking'
]

When typing your list, you accidentally missed the comma that separates "Sculpting" from "Gardening". This mistake introduces a subtle bug into your programs. Because Python doesn’t find the separating comma, it triggers automatic string literal concatenation, joining both items in a single string, 'SculptingGardening'. This type of error may pass unnoticed, causing hard-to-debug issues in your code.

In general, to avoid surprises in your code, you should use explicit string concatenation with + or += whenever you’re working with long strings or strings that contain escape sequences.

Concatenating Strings With StringIO

If you’re working with many strings in a data stream, then StringIO may be a good option for string concatenation. This class provides a native in-memory Unicode container with great speed and performance.

To concatenate strings with StringIO, you first need to import the class from the io module. Then you can use the .write() method to append individual strings to the in-memory buffer:

Python
>>> from io import StringIO

>>> words = ["Hello,", "World!", "I", "am", "a", "Pythonista!"]

>>> sentence = StringIO()
>>> sentence.write(words[0])
6
>>> for word in words[1:]:
...     sentence.write(" " + word)
...
7
2
3
2
12
>>> sentence.getvalue()
'Hello, World! I am a Pythonista!'

Here, you’ll quickly note that a bunch of numbers appear on your screen between operations on the StringIO object. These numbers represent the bytes written or retrieved from the object in each writing or reading operation.

In the above example, you’ve used a finite data stream represented by a list of strings. If you need to work with a potentially infinite data stream, then you should use a while loop instead. For example, here’s a small script that takes words from the user and concatenates them into a sentence:

Python
# sentence.py

from io import StringIO

sentence = StringIO()
while True:
    word = input("Enter a word (or './!/?' to end the sentence): ")
    if word in ".!?":
        sentence.write(word)
        break
    if sentence.tell() == 0:
        sentence.write(word)
    else:
        sentence.write(" " + word)

print("The concatenated sentence is:", sentence.getvalue())

This script grabs the user’s input using the built-in input() function. If the input is a period, an exclamation point, or a question mark, then the loop breaks, terminating the input. Then you check if the buffer is empty by using the .tell() method. Depending on this check, the statement adds the current word only or the word with a leading whitespace.

Here’s how this script works in practice:

Shell
$ python sentence.py
Enter a word (or './!/?' to end the sentence): Hello,
Enter a word (or './!/?' to end the sentence): welcome
Enter a word (or './!/?' to end the sentence): to
Enter a word (or './!/?' to end the sentence): Real
Enter a word (or './!/?' to end the sentence): Python
Enter a word (or './!/?' to end the sentence): !
The concatenated sentence is: Hello, welcome to Real Python!

Cool! Your script works nicely! It takes words at the command line and builds a sentence using StringIO for string concatenation.

Using StringIO to concatenate strings can be an excellent alternative to using the concatenation operators. This tool is handy when you need to deal with a large or unknown number of strings. StringIO can be pretty efficient because it avoids creating intermediate strings. Instead, it appends them directly to the in-memory buffer, which can give you great performance.

StringIO also provides a consistent interface with other file-like Python objects, such as those that open() returns. This means that you can use the same methods for reading and writing data with StringIO as you would with a regular file object.

Using print() to Concatenate Strings

You can also use the built-in print() function to perform some string concatenations, especially to concatenate strings for on-screen messages. This function can help you connect strings with a specific separator:

Python
>>> words = ["Hello,", "World!", "I", "am", "a", "Pythonista!"]

>>> print(*words)
Hello, World! I am a Pythonista!

>>> print(*words, sep="\n")
Hello,
World!
I
am
a
Pythonista!

In these examples, you call print() with an iterable of strings as an argument. Note that you need to use the unpacking operator (*) to unpack your list into multiple separate string objects that will work as individual arguments to print().

The print() function takes a sep argument that defaults to a whitespace. You can use this argument to provide a custom separator for the concatenation. In the second example, you use the newline (\n) escape sequence as a separator. That’s why the individual words get printed one per line.

Another convenient use of print() in the concatenation context is to save the concatenated string to a file. You can do this with the file argument to print(). Here’s an example:

Python
>>> words = ["Hello,", "World!", "I", "am", "a", "Pythonista!"]

>>> with open("output.txt", "w") as output:
...     print(*words, sep="\n", file=output)
...

After running this code, you’ll have output.txt containing your concatenated string, one word per line, because you’ve used the newline escape sequence as a separator. Note that the file argument takes file-like objects. That’s why you use the with statement in the example.

In this example, the as keyword creates the output variable, which is an alias of the file object that open() returns. Then print() concatenates the strings in words using the newline string as a separator and saves the result to your output.txt file. The with statement automatically closes the file for you, releasing the acquired resources.

Conclusion

You’ve learned about various tools and techniques for string concatenation in Python. Concatenation is an essential skill for you as a Python developer because you’ll definitely be working with strings at some point. After learning the most common use cases for each tool or technique, you’re now ready to choose the best approach for your specific problems, empowering you to write more efficient code.

In this tutorial, you’ve learned how to:

  • Understand what string concatenation is
  • Concatenate two strings with the concatenation operators, + and +=
  • Efficiently join multiple strings with the .join() method from str
  • Use alternative concatenation techniques like string literals, StringIO, and print()

With the knowledge that you’ve gained in this tutorial, you’re now on your way to becoming a better Python developer with a solid foundation in string concatenation.

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Leodanis Pozo Ramos

Leodanis is an industrial engineer who loves Python and software development. He's a self-taught Python developer with 6+ years of experience. He's an avid technical writer with a growing number of articles published on Real Python and other sites.

» More about Leodanis

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!

Keep Learning

Related Tutorial Categories: basics best-practices python