Version Control for Real-Time Data
Deploy real-time data products with confidence.
Discover the data version control that lets you build faster, prototype safer, and ship real-time data products more confidently.
At Tinybird, we believe great tools should work the way you work, supplementing your workflow instead of replacing it. As such, we focus on delivering simple, intuitive user experiences.
Today, we are excited to ship more tools that help you deploy real-time data products using the software development processes you already follow:
- Version control with Git repositories. Manage your Tinybird resources with Git. Test, merge, and deploy data projects alongside your existing code.
- Create Releases backed by Git commits. Create a Tinybird Release with a pull request, deploy Releases in preview, and roll back to a prior Release, if needed.
- Develop with Branches and data copies. Create safe spaces to test changes with real data. Copy production data into your Branches to test data pipelines without straining production.
- Integrate and deploy with CI/CD. Deploy with a pull request. Let configurable CI/CD actions handle checks, tests, and deployments automatically.
- Protect production. Prevent unauthorized or untracked changes to your Releases.
“What really makes Tinybird stand out is the simplicity and elegance of the user experience.”
Git Integration
You're already streaming, now you can flow.
Connect your Tinybird Workspace to your Git repository and code like you always do. Manage your entire Tinybird data project as text files, create development Branches, sync Releases to your Git commits, and deploy semantically versioned changes as code.
When you're coding, you want to “flow”. Any deviation from your trusted workflow just slows you down. Now, Tinybird more tightly aligns with the way you already code your production applications.
All of your Tinybird resources are managed as text files and synced with Git, so testing, merging, and deploying Tinybird data projects comes much more naturally.
“When we switched to Tinybird, we ran a PoC and shipped our first feature to production in a month. Since then, we’ve shipped 12 new user-facing features in just a few months. There’s no way we could have done this without Tinybird.”
Releases
Ctrl+Z for data products.
Ship new code with confidence. All changes to your data pipelines are saved in your Release history, so you can roll back in an instant if things don't go as planned.
When you push changes to your production branch, you want assurance that everything will work as expected. If it doesn't, you need to be able to roll back quickly.
Each Tinybird Release is synced to a commit in your Git repository, so your Tinybird deployments are backed by version control. You can deploy Releases in preview to assess how changes will impact production, promote them when you're ready, and roll back to a prior Release with a single click or command.
Branches
Need data? No problem.
Test your changes with real data. Copy production data over to non-production Branches or use data fixtures to test your pipelines from end to end.
Git branches are great for testing code changes, but data complicates things. Now you can copy data from your Live Release into your development Branches, so you can test your code with real, production data.
When you create a new Branch in Tinybird, you have the option to copy over the last partition from your production Data Sources or use data fixtures to add data manually. If you make changes in your source system, you can always append new data to any existing Branch.
“Thanks to Tinybird's support with their Git integration, we now have an amazing CI/CD in place with Tinybird. It's actually the best CI/CD we have in place at the company, period.”
CI/CD
Ship features, not failures.
Make changes, commit them to Git, and deploy them with a pull request. Tinybird's CI/CD integration lets you focus less on DataOps and more on writing code.
Deploying changes to production shouldn't require tribal knowledge; you shouldn't have to scan the company wiki just to add a column to a database table, and you shouldn't need to set up complex checks and balances to avoid breaking things in production.
You should be able to create and merge a PR and trust your CI/CD pipelines to handle the complexities and nuances of production deployments.
Tinybird is tightly integrated with your source control provider's CI/CD. When you create a pull request with Tinybird, pre-supplied YAML files trigger configurable CI/CD actions that perform syntax reviews, check for data quality issues, run regression tests on your APIs, and more.
ci / Deploy to CI Branch
- Set up job
- Run actions/checkout@main
- Run actions/setup-python@v5
- Validate input
- Set environment variables
- Install Tinybird CLI
- Tinybird version
- Check all the data files syntax
- Check auth
- Try delete previous Branch
- Create new test Branch with data
- Deploy changes to the test Branch
- Custom deployment to the test Branch
- Post Run actions/setup-python@v5
- Post Run actions/checkout@main
- Complete job
User Access Control
Production is safe and sound.
Protect your main branch from unwanted changes. Assign roles and disable deployments for read-only users.
As with all of your code, version control defines the source of truth. With Tinybird's new "Viewer" role, you can give read-only access when inviting users to your Tinybird Workspaces to prevent unauthorized and untracked changes.
“I've worked with quite a few different data stacks, and I truly don't think we could have shipped what we have without Tinybird. We have achieved in a few days what would have taken ages with any other tooling.”
This should feel familiar
Tinybird is a platform for developer productivity. With Tinybird, data engineers and software developers can collaborate on and very quickly deploy real-time data pipelines from ingestion to publication with minimal friction and minimal risk.
In some ways, Tinybird is like dbt, a tool familiar to many data engineers. dbt was built upon the belief that "anyone on the data team should be able to safely contribute to production-grade data pipelines".
We align with this belief, albeit with some modifications: Anyone on the data and software teams should be able to safely contribute to and deploy production-grade, real-time data pipelines for operational and user-facing analytics use cases.
Tinybird and dbt solve fundamentally different data problems, and dbt's batch-oriented technical model doesn't work for real-time data needs. Still, we believe that all data development, whether in real-time or batch, should be fast, collaborative, and safe.
Tinybird now includes a raft of features and functionality that simplify and accelerate how you work with real-time data. Our primary mission is and always has been to give developers tools they love to help them turn data at any scale into real-time insights, actions, and business value.
With these new features, we meet you where you are, bringing to the fore those tried and true workflows - version control, branching, testing, and CI/CD - that make you more productive and more confident in your work.
Notes from the product team
Why we built versioning into Tinybird
The lead Engineer and Product Manager for Tinybird's Git integration discuss these new tools, why they matter, and how they improve the way you work with real-time data in production.
Documentation
Want to learn how to version control your data projects?
Dive into the Tinybird docs and discover a more delightful way of working with real-time data.
Build fast data
products, faster.
Calle del Dr. Fourquet, 27
28012 Madrid
41 East 11th Street 11th floor
New York, NY 10003
Calle del Dr. Fourquet, 27
28012 Madrid
41 East 11th Street 11th floor
New York, NY 10003