DEV Community: TechiesJournal

Pulumi and Terraform Can Work Together: What Actually Changes for Infrastructure Teams?

TechiesJournal — Sun, 15 Nov 2026 07:12:40 +0000

Pulumi’s 2026 interoperability improvements let organizations retain Terraform workflows, share infrastructure modules across languages and run HCL through the Pulumi engine. The options reduce some migration barriers, but each has different implications for state, execution and resource ownership.

An infrastructure team may have years of work invested in Terraform modules, deployment pipelines and production state. Moving to another infrastructure-as-code platform can mean rewriting configuration, changing delivery processes and validating resources that already work.

Pulumi’s August 2026 release expands the ways teams can use Terraform, OpenTofu (an open-source Terraform-compatible implementation) and Pulumi together without replacing everything at once. The release brings together three capabilities: Terraform state and execution through Pulumi Cloud, Terraform module interoperability across languages, and HCL (HashiCorp Configuration Language, the language of Terraform configuration files) as a Pulumi programming language. These are different approaches. Understanding that distinction is more useful than treating the release as a single migration feature.

Three capabilities, three different changes

Capability	What changes	What remains important
Terraform backend and remote execution	Terraform state and optional execution move to Pulumi Cloud	Terraform or OpenTofu workflows remain, with remote execution subject to Pulumi Cloud’s execution model
Terraform module interoperability	Existing modules can be published, consumed and reused across supported languages	Module execution, dependencies and compatibility still require validation
Pulumi HCL runtime	HCL configuration runs on the Pulumi engine	Pulumi state and resource ownership replace the Terraform execution model

The difference begins with two concepts. Configuration describes the infrastructure a team wants. State records the real resources associated with an infrastructure management workflow.

Moving state storage does not necessarily change the engine that applies configuration. Running the same HCL through a different engine, however, changes more than the location of a state file.

Source: Pulumi, Full Support for Terraform State, Modules and HCL, August 2026.

1. Keep Terraform and use Pulumi Cloud for state

A team can continue writing HCL and running familiar Terraform or OpenTofu commands while storing state in Pulumi Cloud. The backend provides centralized state management, locking, access controls, update history and visibility into managed resources. This arrangement does not turn Terraform-managed resources into Pulumi-engine-managed resources. The existing workflow can remain largely intact.

Pulumi Cloud also offers remote execution, but this must be distinguished from using it only as a state backend. With local execution, the Terraform or OpenTofu CLI runs operations in the team’s own environment while Pulumi Cloud stores the state. With remote execution, the CLI sends configuration to Pulumi Cloud, where managed infrastructure performs the plan or apply.

An important implementation detail is that Pulumi Cloud’s remote runner uses OpenTofu as its execution engine, even when the operation was initiated through the Terraform CLI. The managed OpenTofu version is currently controlled by Pulumi Cloud rather than selected by the customer.

Remote execution also has specific limitations, including unsupported -var flags and the absence of configurable custom executor images. Those differences matter for teams whose existing pipelines depend on particular Terraform versions, command-line arguments or runner customization.

Interactive figure: view it in the original article

Sources: Pulumi, Store Terraform State in Pulumi Cloud and Remote Execution for Terraform and OpenTofu.

2. Share Terraform modules across languages

Terraform modules often represent established infrastructure standards. A platform team might maintain modules for networks, databases, storage or access controls. Rewriting these components solely because another team wants to use Python or TypeScript creates work and introduces potential differences. Pulumi’s interoperability features provide two related approaches.

Module registry and generated components. Teams can publish Terraform modules to Pulumi Cloud’s private registry. The platform can expose them to Terraform and OpenTofu consumers and generate typed components for Pulumi-supported languages.

Any HCL Module integration. Pulumi programs can use existing Terraform modules through an integration that uses OpenTofu to execute the module, maps supplied inputs to Terraform variables and exposes the results as Pulumi outputs.

The Any HCL Module mechanism manages its module resources through the Pulumi state backend. It is not the same thing as independently running an existing Terraform root configuration with its original state. Generated SDKs can provide typed inputs and outputs, while dynamic module loading offers more flexibility at the expense of some type safety.

The practical benefit is reusing tested infrastructure definitions across teams with different programming preferences. The remaining responsibility is compatibility testing. Modules with complex provider dependencies or execution assumptions still need validation in their intended environment.

Source: Pulumi, Use Terraform Modules in Pulumi.

3. Run HCL through the Pulumi engine

The third capability changes the infrastructure execution model. Pulumi supports HCL as a language for its engine. Teams can retain familiar .tf configuration syntax while using Pulumi’s resource lifecycle and state management. This may reduce the effort required to evaluate Pulumi without rewriting an entire configuration into another programming language.

However, Terraform state does not automatically become Pulumi state. Existing resources must be brought under Pulumi management through an explicit import process.

Pulumi documents the following mechanism for HCL migrations:

pulumi import --from hcl <statefile>

The import reads an existing Terraform state file and maps resources into the Pulumi stack. It is a migration operation, not an instruction to run two independent engines against the same infrastructure resources.

Before using this approach, teams must validate provider mappings, resource identities, dependencies and the resulting preview. They should avoid simultaneous management of the same resources by independent state systems.

There are also compatibility differences. For example, Terraform backend and certain other configuration settings are accepted but ignored with warnings because Pulumi manages state independently. A Terraform cloud block is not accepted. Provisioner connection support is limited, including no WinRM support.

Pulumi’s engineering team reported in Emulating Terraform on Pulumi’s Engine that its HCL interpreter ran 56 of the 58 most-used Terraform AWS modules, which is 96%, and that the failures were due to ephemeral resources. That is a vendor-reported result for a particular set of AWS modules, not evidence that 96% of every organization’s Terraform configurations will migrate successfully.

Sources: Pulumi, HCL Documentation, HCL Language Reference, and Emulating Terraform on Pulumi’s Engine.

What should teams choose?

Return to the infrastructure team from the opening, and suppose it operates an established Terraform environment but wants to introduce Pulumi for new application infrastructure. This is a hypothetical team, used here to compare the options. There is no requirement to migrate everything at once.

Existing requirement	Approach to evaluate
Better centralized state management	Pulumi Cloud Terraform backend
Managed Terraform or OpenTofu execution	Pulumi Cloud remote execution, after validating its OpenTofu-based runner
Shared Terraform modules across language ecosystems	Pulumi Cloud module registry and generated components
Terraform modules inside Pulumi programs	Any HCL Module integration
HCL configuration managed by the Pulumi engine	Pulumi HCL with explicit state and resource import
Stable existing infrastructure with no compelling migration need	Retain the current Terraform workflow

The right choice depends on the operational problem. A new state backend may help a team that needs centralized controls. Shared modules may help organizations supporting multiple programming languages. A different execution engine may make sense when a team wants Pulumi’s broader programming model.

None of those benefits automatically justifies migrating an established, working infrastructure estate.

Operational boundaries that still matter

Three areas deserve particular attention before production adoption.

Resource ownership. Each resource must have one clearly defined management path. Teams should know which engine applies changes and which state system records the resource.

Policy and drift detection. Pulumi Cloud can run audit policies against Terraform-managed stacks, which its documentation lists for the Essentials, Pro and Enterprise editions. Preventative policies that block a violating apply are available through remote execution, documented for the Pro and Enterprise editions. However, automated drift detection is not currently available for Terraform-managed stacks. Organizations may need scheduled plans or another existing monitoring process.

Cost and execution constraints. Terraform resources stored in Pulumi Cloud count toward resources under management. Remote execution consumes billable workflow minutes. Teams should also account for provider versions, runner limitations and existing deployment integration requirements. Provider upgrades are a common source of such constraints, as The Terraform AWS Provider Keeps Growing: Is Your Upgrade Process Ready? explains.

These are not reasons to reject interoperability. They are part of determining whether a particular integration improves the organization’s current operating model.

Sources: Pulumi, Remote Execution for Terraform and OpenTofu and Store Terraform State in Pulumi Cloud.

A practical adoption path

Teams can evaluate interoperability without launching a broad migration. If the objective is state management, test the backend with a representative non-production Terraform stack. Confirm that initialization, state migration, locking and planning work as expected.

If the objective is module reuse, select a module with meaningful dependencies and verify its behavior through the chosen integration. If the objective is Pulumi-engine adoption, test the HCL runtime with a controlled resource import and verify that the resulting preview contains no unintended changes.

These tests answer different questions. A successful backend migration does not prove HCL runtime compatibility. A reusable module does not prove the entire Terraform estate can move without redesign. The goal should be a verified improvement in the current workflow, not simply adoption of another tool.

What the release actually changes

Pulumi’s August 2026 interoperability improvements give infrastructure teams more ways to preserve existing Terraform investments while adopting selected Pulumi capabilities. They do not make Terraform, OpenTofu and Pulumi identical.

The important distinction is between where state is stored, how modules are reused and which engine manages resources. Organizations that understand those boundaries can adopt interoperability incrementally while maintaining clearer ownership and more predictable infrastructure changes.

References and further reading

Pulumi, Full Support for Terraform State, Modules and HCL. August 2026.
Pulumi, Now Generally Available: Store Terraform State in Pulumi Cloud. March 5, 2026.
Pulumi, Store Terraform State in Pulumi Cloud. Official technical documentation.
Pulumi, Remote Execution for Terraform and OpenTofu. Official execution behavior and limitations.
Pulumi, Use Terraform Modules in Pulumi. Module interoperability.
Pulumi, HCL Language Documentation. Engine and state compatibility.
Pulumi, HCL Language Reference. Unsupported features and migration requirements.
Pulumi, Emulating Terraform on Pulumi’s Engine. August 4, 2026.

Source verification: November 5, 2026. Capabilities and compatibility claims are based on Pulumi’s published documentation and engineering reports. TechiesJournal has not independently tested these production migrations. The 96 percent figure was checked against the footnote of Pulumi’s own post.

Originally published on TechiesJournal.

PostgreSQL and pgvector: When Should AI Applications Use a Separate Vector Database?

TechiesJournal — Sun, 15 Nov 2026 06:22:18 +0000

An AI application can keep business records, embeddings and searchable documents in PostgreSQL, or introduce a dedicated vector system. The right decision depends on retrieval quality, access controls, scaling requirements and the cost of keeping multiple data stores synchronized.

An organization building an AI knowledge assistant may already store its documents, users and permissions in PostgreSQL. To make those documents searchable by meaning, the application needs embeddings. These are numerical representations of content that allow software to find semantically related passages.

One option is to store embeddings alongside application data using the pgvector extension. Another is to maintain a separate vector-search system. Both architectures can support retrieval-augmented generation (RAG).

The question is whether a separate database solves a measurable problem or introduces complexity that the application does not yet need.

What pgvector adds to PostgreSQL

PostgreSQL already provides relational storage, transactions, joins, indexing and full-text search. pgvector adds vector data types, similarity operators and approximate nearest-neighbor indexes, which find close matches quickly without checking every vector.

An application can store document chunks, their embeddings and associated metadata in PostgreSQL. When a user asks a question, the application creates an embedding of that question and searches for similar document embeddings. The retrieved passages can then provide context to a language model generating an answer.

pgvector does not generate embeddings, enforce the correctness of model-generated answers or automatically determine whether a document is safe for a particular user to read. Those remain responsibilities of the application and its security architecture.

Importantly, pgvector is an extension, not a PostgreSQL 19 feature. Its existing capabilities work across supported PostgreSQL versions, and the pgvector README lists support for PostgreSQL 13 and later, so check which versions your release supports. For what changes in PostgreSQL itself, see our article on PostgreSQL 19 and its release candidate.

Source: pgvector official documentation.

How an enterprise RAG application retrieves documents

Consider a hypothetical internal knowledge assistant used by Finance and Human Resources employees. The organization stores documents and access metadata in PostgreSQL. Each document is divided into searchable passages, with an embedding associated with each passage.

A typical retrieval operation has several stages.

1. Authenticate the user. The application establishes the user’s identity and authorized organizational context.

2. Determine the permitted document scope. The server identifies which documents the user is entitled to access, using trusted authorization data rather than user-supplied instructions.

3. Generate the question embedding. An embedding model converts the user’s question into the same compatible vector representation used for the stored passages.

4. Retrieve matching passages. PostgreSQL searches for semantically similar content while applying the appropriate document-access conditions.

5. Enforce the access boundary. The application verifies that only permitted content can be passed to the model. Database-side row-level security can provide another enforcement layer where correctly configured.

6. Generate the response. The language model uses authorized passages as supporting context. The application can retain source references for traceability.

Interactive figure: view it in the original article

Suppose a Finance employee asks about an expense-approval policy. A Human Resources document might be semantically similar, but similarity does not make that document authorized. The database query and surrounding application controls must prevent restricted material from entering the model’s context.

PostgreSQL row-level security

PostgreSQL row-level security (RLS) can restrict which records a database role may read or modify. It can help enforce tenant or document boundaries independently of ordinary application filtering.

However, enabling RLS does not automatically make every connection safe. Superusers and roles with BYPASSRLS bypass policies. Table owners normally bypass RLS unless FORCE ROW LEVEL SECURITY is configured. Applications also need to establish authorization context safely, especially when connections are pooled or reused.

RLS should be tested with realistic user roles, including negative tests proving that unauthorized documents cannot be retrieved. An ordinary WHERE tenant_id = ... clause is useful for query filtering, but should not be treated as a complete security boundary when the calling application can alter or omit it.

Source: PostgreSQL, Row Security Policies.

Exact search and approximate indexes solve different problems

By default, pgvector performs exact nearest-neighbor search. This finds the nearest matching vectors under the selected distance measure, providing a useful baseline against which approximate results can be evaluated. As vector collections and request volumes grow, exact search may become more expensive.

pgvector supports two principal approximate indexing methods.

Index	How it works	Important trade-off
HNSW	Organizes vectors into a navigable graph	Strong speed and recall characteristics for many workloads, with additional build time and memory requirements
IVFFlat	Groups vectors into lists and searches selected lists	Requires suitable training data and tuning of search breadth

Approximate indexes can return different results from exact search. That is why latency alone is an insufficient performance measure. Teams should also measure recall, which indicates how many expected nearest-neighbor results were retrieved compared with an exact-search baseline.

For an AI assistant, a faster result may be less useful if an important document is consistently omitted.

Source: pgvector, Indexing.

Filtering can affect retrieval quality

Enterprise searches rarely depend on vector similarity alone. Results may also be filtered by organization, document type, project or access scope. With approximate indexes, PostgreSQL can apply filtering after the index scan has selected candidates, and this can return fewer results than requested.

For example, if a search examines 40 candidates and only 10% satisfy a category filter, approximately four may remain on average. That is an illustration of selectivity, not a guaranteed outcome for every search.

pgvector introduced iterative index scans in version 0.8.0 on October 30, 2024. Iterative scans can continue examining more index candidates when filtering removes too many results, subject to configured resource and scan limits. Other approaches include ordinary PostgreSQL indexes, partial vector indexes and table partitioning, depending on query patterns. These mechanisms should be selected through measurement.

Tenants that share one approximate index can also affect each other. The pgvector documentation says vectors from one tenant can change recall and speed for other tenants, and recommends list partitioning or separate tables for tenant isolation.

A correctly enforced authorization filter must never be weakened merely to increase the number of search results. A retrieval system returning fewer authorized passages may produce a less complete answer. Returning unauthorized passages is a different and more serious failure.

Sources: pgvector changelog, pgvector filtering guidance and pgvector multitenancy.

Hybrid search combines different forms of relevance

Semantic similarity is useful when users describe a concept without using the exact wording found in a document. Keyword search remains important for identifiers, product names, policy numbers and technical terms.

PostgreSQL supports full-text search through facilities such as tsvector and tsquery. Applications can combine those results with pgvector similarity search. For example, a knowledge assistant searching for a specific incident number may benefit from exact text matching, while a broader troubleshooting question may benefit from semantic retrieval.

Ranked results can be combined using techniques such as reciprocal rank fusion, which merges two ranked lists by position, then optionally reranked using another relevance model. However, hybrid search does not guarantee better retrieval in every application. Teams should evaluate representative questions, expected documents and final ranking quality.

Sources: PostgreSQL full-text search and pgvector hybrid search.

When does a separate vector database become useful?

A dedicated vector system may become worthwhile when an application’s retrieval requirements exceed what its existing PostgreSQL environment can support effectively. Examples include independently scaling vector-search traffic, specialized distributed indexing, very large search collections or retrieval features that are unavailable in the chosen PostgreSQL deployment.

But moving embeddings to another system creates additional responsibilities.

Consider a document update. The application changes the source document in PostgreSQL. It must also regenerate affected embeddings and update the vector index. Until synchronization finishes, searches may retrieve outdated content.

Deletion creates a related problem. Removing a document from the transactional database must also remove or invalidate its vector representations.

Permission changes are particularly important. If a user’s access is revoked, stale access metadata in another search system must not allow restricted content to reach the model.

An architecture with two data stores therefore needs a reliable way to propagate changes, handle failures, retry safely and determine when search data is current. Change-data capture, which streams changes out of the database, transactional outbox patterns, which record the change event in the same transaction as the data, and event processing can help, but each requires implementation and operational ownership. The authoritative permission check should remain enforceable even when the search index is delayed.

These synchronization boundaries are part of the cost of adopting a separate vector system.

Embedding models introduce another migration boundary

An embedding is generated using a particular model and representation. Changing the embedding model may change vector dimensions, distributions and similarity behavior. Embeddings produced by incompatible models should not be combined as though their distances were directly comparable.

Organizations should record embedding-model identifiers, versions and relevant generation settings. A model migration may require generating new embeddings, rebuilding indexes and evaluating retrieval quality before switching production queries. During a transition, old and new representations may need to coexist.

pgvector also has indexing limits that depend on the representation. Its documented HNSW support includes vector values up to 2,000 dimensions and halfvec values up to 4,000 dimensions. These limits concern the relevant index representation, not a universal restriction on every possible storage or search strategy. The application’s embedding-model choice therefore affects index design.

Source: pgvector, Vector Types and Indexing Limits.

How should a team choose?

A useful evaluation compares the existing PostgreSQL approach with a separate vector system using the same representative workload.

Evaluation area	Question to answer
Retrieval quality	Are expected relevant passages consistently found?
Authorization	Can unauthorized documents ever reach the model context?
Performance	Does search latency meet the target under realistic concurrency?
Filtering	Do selective tenant and metadata conditions preserve acceptable recall?
Updates and deletions	How quickly does the search index reflect source changes?
Operations	What are the indexing, backup, monitoring and recovery requirements?
Scaling	Can retrieval demand grow without harming transactional workloads?
Cost	What is the combined infrastructure and maintenance expense?

The comparison should not assume that the most specialized technology is automatically the most suitable. A small or moderately sized application may find that PostgreSQL with pgvector meets its needs with fewer systems to manage. A larger application may justify an independent search tier when workload isolation, specialized features or independent scaling delivers measurable benefits. Those outcomes are workload-dependent and should be established through tests, not generic database-size thresholds.

Operations also include keeping the extension patched. The pgvector changelog records fixes for possible index corruption during HNSW vacuuming in version 0.8.3 on June 17, 2026, and for buffer overflows during IVFFlat index builds in versions 0.8.6 on July 29 (on 32-bit systems) and 0.8.7 on October 1, 2026. Teams should run a current patch release and test index maintenance, such as vacuuming and rebuilds, before relying on an index in production.

Source: pgvector, Changelog.

What the architecture decision comes down to

PostgreSQL and pgvector allow application records, embeddings, metadata and text search to remain within one database environment. This can simplify AI applications that already depend heavily on relational data. A separate vector system may provide valuable capabilities, but requires attention to synchronization, deletions, authorization and operational ownership.

Neither architecture automatically guarantees search relevance or secure retrieval.

Start with verified retrieval and access-control requirements. Use PostgreSQL with pgvector when it meets them effectively, and introduce a separate vector system when measurements justify the additional complexity.

References and further reading

pgvector, Official Repository. Supported vector types, similarity search, indexing, filtering and multitenancy.
pgvector, Changelog. Version history, iterative index scans and patch releases.
PostgreSQL, Row Security Policies. Database authorization behavior and bypass conditions.
PostgreSQL, Full Text Search. Keyword retrieval and ranking.
PostgreSQL, Full Text Search Introduction. The tsvector and tsquery types.
PostgreSQL, EXPLAIN. Execution-plan inspection and performance evaluation.
pgvector, Release and Security History. Extension releases and maintenance updates.

Sources verified November 6, 2026. pgvector releases often, and the 0.8.7 release on October 1, 2026 fixed an IVFFlat index build bug and is not presented as introducing the architectural capabilities discussed here, so check the sources above before you act. Performance and architecture guidance are analytical recommendations, not independent TechiesJournal benchmark results.

Originally published on TechiesJournal.