# PEFT review

> PEFT is a Hugging Face Python library for adapting large pretrained models by fine-tuning only a small number of extra parameters.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/peft/

## TL;DR

PEFT is a Python library from Hugging Face that implements parameter-efficient fine-tuning methods such as LoRA, IA3, and soft prompts. It is for developers who want to adapt large models with lower compute and storage costs. PEFT integrates with Transformers, Diffusers, and Accelerate, and the resulting adapter checkpoints are often only a few MBs.

## PEFT at a glance

- **Category:** [Inference and training](https://index.devtoolwatch.com/categories/inference-and-training/)
- **Maintained by:** Hugging Face (organization, [@huggingface](https://github.com/huggingface) on GitHub)
- **License:** Apache 2.0
- **Language:** Python, Makefile, Dockerfile
- **Repository:** [github.com/huggingface/peft](https://github.com/huggingface/peft)
- **Website:** [huggingface.co/docs/peft](https://huggingface.co/docs/peft)
- **PyPI package:** [peft](https://pypi.org/project/peft/) 0.21.2
- **Latest release:** [v0.21.2](https://github.com/huggingface/peft/releases/tag/v0.21.2) on GitHub, Oct 1, 2026
- **First published:** Nov 25, 2022

## What PEFT does

- PEFT wraps a base model and a configuration such as LoraConfig with get_peft_model, so only a small fraction of parameters is trained.
- PEFT implements many adaptation methods, including LoRA and its variants, AdaLoRA, IA3, LoHa, LoKr, OFT, prefix tuning, and prompt tuning.
- PEFT saves trained adapters as small checkpoints, often a few MBs instead of GBs, which can be loaded onto the base model for inference.
- PEFT can be combined with quantization, for example QLoRA, to fine-tune LLMs on consumer GPUs.
- PEFT applies to diffusion models, for example training Stable Diffusion with LoRA through the Diffusers integration.
- PEFT supports RLHF and DPO training of LLMs through the TRL library.

### How it works

PEFT runs as a Python library on top of PyTorch. You load a model with Transformers, define a PEFT configuration, and wrap the model with get_peft_model before training. For inference, PeftModel.from_pretrained loads a saved adapter onto the base model. Transformers models can also call add_adapter, load_adapter, and set_adapter directly. Accelerate handles distributed training and inference.

## How to install PEFT

**With pip**

```
pip install peft
```

### Requirements

- Python 3.10.0 or later.
- PyTorch, which the library is built on.
- Tensor Parallel support in v0.21.1 requires Transformers 5.17.0 or later.

### First steps

1. Install PEFT with pip, then load a base model with Transformers AutoModelForCausalLM.from_pretrained.
2. Create a LoraConfig, setting values such as r, lora_alpha, and task_type, and wrap the model with get_peft_model.
3. Call print_trainable_parameters to check how many parameters will be trained, then train, for example with the Transformers Trainer.
4. Save the adapter with save_pretrained, and load it later for inference with PeftModel.from_pretrained.

## Who PEFT is for

- Developers fine-tuning LLMs on limited GPU memory or consumer hardware.
- Teams adapting one base model to many tasks or datasets who want small per-task checkpoints.
- Users of Transformers, Diffusers, Accelerate, or TRL who want adapter-based training.
- Researchers who need reference implementations of many parameter-efficient fine-tuning methods.

### Limitations

- The Transformers integration does not include all PEFT features, such as merging the adapter into the base model.
- Some models are not supported out of the box, so you must configure the model manually to enable PEFT.
- PEFT is tied to the PyTorch and Hugging Face ecosystem, and some features depend on specific Transformers versions.

## Usage data (the week of Oct 25–31, 2026)

- Board rank: #303 overall, #34 in Inference and training, momentum score 31
- Downloads that week: 2.2M (+6% vs 4-week average)
- New GitHub stars per week (last 12 weeks): Aug 9: 45; Aug 16: 34; Aug 23: 31; Aug 30: 35; Sep 6: 32; Sep 13: 39; Sep 20: 29; Sep 27: 29; Oct 4: 30; Oct 11: 28; Oct 18: 54; Oct 25: 28

## PEFT alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [dsh-our-free-model](https://index.devtoolwatch.com/tools/ebony-vinyl-dsh-our-free-model/) | 54.3k | – |
| #2 | [magpie](https://index.devtoolwatch.com/tools/yetone-magpie/) | 26.9k | – |
| #3 | [Strata](https://index.devtoolwatch.com/tools/niko1221-strata/) | 37.1k | – |
| #4 | [edge0](https://index.devtoolwatch.com/tools/edge0-ai-edge0/) | 8.9k | – |
| #5 | [OmniRoute](https://index.devtoolwatch.com/tools/diegosouzapw-omniroute/) | 83.2k | 106k |
| #6 | [CLIProxyAPI](https://index.devtoolwatch.com/tools/router-for-me-cliproxyapi/) | 57.1k | – |

## PEFT FAQ

### Is PEFT free and open source?

Yes. PEFT's source code is on GitHub (huggingface/peft) under the Apache 2.0 license, so you can use it for free. Its website also links to a pricing page (https://huggingface.co/pricing) for any paid plans.

### How do I install PEFT?

Run `pip install peft`. The peft package needs Python 3.10.0 or newer.

### Is PEFT still maintained?

Yes. The huggingface/peft repository was last updated on Oct 21, 2026, and the latest release, v0.21.2 on GitHub, came out on Oct 1, 2026.

### How popular is PEFT?

PEFT has 21.9k GitHub stars as of Nov 1, 2026. It gained 28 new stars in the week of Oct 25–31, 2026, against a 12-week average of 35 a week. Its main package had 2.2M downloads that week, +6% against its own 4-week average. That made it #34 among LLM inference and training tools on the Devtoolwatch Board.

### What are the best alternatives to PEFT?

Other LLM inference and training tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: dsh-our-free-model (#1), magpie (#2), Strata (#3), edge0 (#4), OmniRoute (#5). The alternatives table above has their numbers.

### Is LoRA the same as PEFT?

No. PEFT is a library that implements many parameter-efficient fine-tuning methods, and LoRA is one of them. PEFT also includes LoRA variants such as DoRA, along with IA3, prefix tuning, prompt tuning, OFT, and others.

### Can PEFT be used with QLoRA?

Yes. PEFT can be combined with quantization to cut memory use further. The README links a guide to fine-tuning Llama-2-7b with QLoRA and TRL on a 16GB GPU.

### How does PEFT compare with full fine-tuning?

PEFT trains only a small number of extra parameters instead of all of them. This lowers compute and storage costs. The sources say recent PEFT methods reach performance comparable to fully fine-tuned models, with checkpoints of a few MBs instead of GBs.

### Can PEFT be used for supervised fine-tuning and RLHF?

Yes. PEFT works with the Transformers Trainer for supervised training and with the TRL library for RLHF and DPO, including supervised finetuning, reward modeling, and RL finetuning examples.

### Does PEFT work with diffusion models?

Yes. Through the Diffusers integration, PEFT can train Stable Diffusion with LoRA. This lowers memory needs and produces small final checkpoints.

## Sources

1. [huggingface/peft on GitHub](https://github.com/huggingface/peft)
2. [PEFT README](https://github.com/huggingface/peft/blob/main/README.md)
3. [peft on PyPI](https://pypi.org/project/peft/)
4. [PEFT website](https://huggingface.co/docs/peft)
5. [Release v0.21.2 (Oct 1, 2026)](https://github.com/huggingface/peft/releases/tag/v0.21.2)
6. [Release v0.21.1 (Sep 29, 2026)](https://github.com/huggingface/peft/releases/tag/v0.21.1)
7. [Release v0.21.0 (Sep 15, 2026)](https://github.com/huggingface/peft/releases/tag/v0.21.0)
8. [Hacker News: Qwen 3.8 27B](https://news.ycombinator.com/item?id=49299605)
9. [Hacker News: Kimi-K3 on HuggingFace](https://news.ycombinator.com/item?id=49065752)
10. [Hacker News: DeepSeek-v3.2: Pushing the frontier of open large language models [pdf]](https://news.ycombinator.com/item?id=46108780)
11. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)