CodeT5

CodeT5 is a family of open-source code large language models from Salesforce Research, built on the T5 transformer architecture to understand and generate source code.

The original model, presented at EMNLP 2021, is an encoder-decoder pretrained on 8.35 million code functions in eight languages, including Python. Its identifier-aware pretraining teaches the model which tokens are developer-assigned names and how to recover them when masked. That design suits code summarization, code translation, defect detection, and clone detection.

To see what that objective looks like, switch between plain span masking and CodeT5’s identifier masking below, then click any sentinel to recover the name the model has to predict:

CodeT5+ followed in 2023 with checkpoints from 220 million to 16 billion parameters that run in encoder-only, decoder-only, or encoder-decoder mode. The instruction-tuned InstructCodeT5+ 16B scored 35.0 percent on the HumanEval pass@1 benchmark and is released for research use only.

Python developers pull the checkpoints from Hugging Face with the transformers library through AutoTokenizer and AutoModelForSeq2SeqLM, then fine-tune them on their own code. Salesforce also built a VS Code plugin demo for Apex developers that covers text-to-code generation, function-level autocompletion, and code summarization.

Salesforce archived the CodeT5 repository in June 2026, and no new models followed the May 2023 CodeT5+ release. The checkpoints stay downloadable from Hugging Face, so existing pipelines keep working, but the family is frozen. Open-weight peers include StarCoder, Code Llama, and CodeGeeX.

Official website: github.com/salesforce/CodeT5

Related Resources

Tutorial

Hugging Face Transformers: Leverage Open-Source AI in Python

As the AI boom continues, the Hugging Face platform stands out as the leading open-source model hub. In this tutorial, you'll get hands-on experience with Hugging Face and the Transformers library in Python.

intermediate ai

For additional information on related topics, take a look at the following resources:

• How to Use Ollama to Run Large Language Models Locally (Tutorial)
• Document Your Python Code and Projects With ChatGPT (Tutorial)
• Write Unit Tests for Your Python Code With ChatGPT (Tutorial)
• Prompt Engineering: A Practical Example (Tutorial)
• Build an LLM RAG Chatbot With LangChain (Tutorial)
• Hugging Face Transformers (Quiz)
• How to Use Ollama to Run Large Language Models Locally (Quiz)
• Practical Prompt Engineering (Quiz)
• First Steps With LangChain (Course)
• Build an LLM RAG Chatbot With LangChain (Quiz)