Building an Idempotent TallyPrime-to-Odoo Enterprise Pipeline in Python

Engineering lessons from migrating 87,408 historical vouchers and synchronizing Odoo Community 11 and 15 with Odoo Enterprise.

tags: odoo, python, architecture, datamigration

Migrating accounting data is rarely just an ETL problem.

It becomes a distributed-systems problem when:

• Historical accounting data lives in TallyPrime
• Operations continue in two independent Odoo Community databases
• The Community databases run different Odoo versions
• The target is a multi-company Odoo Enterprise environment
• The old operational systems cannot be switched off
• New transactions must continue synchronizing after migration
• Retries must never create duplicate accounting records

This article explains the engineering patterns we used while migrating more than 14 years of accounting history for a UAE-based leather goods manufacturer and repair business.

The completed implementation included:

• Odoo Community 11
• Odoo Community 15
• TallyPrime 4.1
• Odoo Online Enterprise Custom 19.4
• Two legal companies in one Enterprise database
• 87,408 eligible historical vouchers migrated and validated
• 79,157 eligible ledger lines for one company
• A Python synchronization service running every 20 seconds
• Controlled reverse synchronization for selected workflows

This is not a walkthrough of the client's production code. The examples below are simplified, sanitized representations of the architecture and engineering decisions.

Table of contents

• Start with invariants, not APIs
• Separate source adapters from accounting logic
• Normalize records into a canonical model
• Build the TallyPrime migration as a staged pipeline
• Use stable source identities
• Design for at-least-once delivery
• Handle the create-before-checkpoint crash window
• Use restart-safe incremental checkpoints
• Keep multi-company mappings isolated
• Treat reverse synchronization as an allowlisted projection
• Validate accounting at multiple levels
• Make rollback ownership-aware
• Add observability around business outcomes
• Results and engineering lessons

Start with invariants, not APIs

The first mistake in a complex integration is starting with the API documentation.

Before deciding how to read or write records, we defined the conditions that had to remain true.

The important invariants were:

1. A source accounting document must not produce more than one target document.
2. Records from one legal company must never be assigned to the other company.
3. A job must be safe to retry after a timeout or process restart.
4. Every target record must be traceable to its source.
5. Unsupported or ambiguous data must be reported, not silently discarded.
6. Migration-created records must be distinguishable from genuine production records.
7. Reverse synchronization must only update explicitly approved fields.
8. A technically successful import must still pass accounting validation.

These invariants shaped the architecture more than the choice between XML-RPC, XML or JSON-2.

Separate source adapters from accounting logic

The two operational systems ran Odoo Community 11 and Odoo Community 15.

Although both were Odoo systems, we could not treat them as identical data sources. Models, available fields, customizations and workflow assumptions differed.

We introduced a version-aware adapter boundary.

from datetime import datetime
from typing import Protocol, Sequence


class CommunityAdapter(Protocol):
    def fetch_changed_invoices(
        self,
        changed_after: datetime,
        limit: int,
    ) -> Sequence[dict]:
        ...

    def fetch_partner(self, partner_id: int) -> dict:
        ...

    def fetch_product(self, product_id: int) -> dict:
        ...

    def update_payment_status(
        self,
        invoice_id: int,
        status: str,
    ) -> None:
        ...

The two implementations handled source-specific behavior.

class Odoo11Adapter:
    def __init__(self, xmlrpc_client):
        self.client = xmlrpc_client

    def fetch_changed_invoices(self, changed_after, limit):
        raw_records = self.client.search_read(
            model="account.invoice",
            domain=[("write_date", ">=", changed_after)],
            fields=self._invoice_fields(),
            limit=limit,
            order="write_date,id",
        )
        return [self._translate_invoice(record) for record in raw_records]


class Odoo15Adapter:
    def __init__(self, xmlrpc_client):
        self.client = xmlrpc_client

    def fetch_changed_invoices(self, changed_after, limit):
        raw_records = self.client.search_read(
            model="account.move",
            domain=[
                ("move_type", "=", "out_invoice"),
                ("write_date", ">=", changed_after),
            ],
            fields=self._invoice_fields(),
            limit=limit,
            order="write_date,id",
        )
        return [self._translate_invoice(record) for record in raw_records]

The model names above illustrate an important compatibility issue.

An invoice represented through an older accounting model cannot simply be queried and processed with assumptions taken from a newer Odoo version.

The adapters absorbed those differences. The rest of the integration worked with a common internal representation.

Normalize records into a canonical model

We did not allow source-specific dictionaries to flow through the complete integration.

Instead, every adapter translated its records into a canonical accounting model.

from dataclasses import dataclass
from datetime import date, datetime
from decimal import Decimal
from typing import Literal


SourceSystem = Literal["tally", "odoo11", "odoo15"]

DocumentType = Literal[
    "customer_invoice",
    "vendor_bill",
    "credit_note",
    "debit_note",
    "journal_entry",
    "payment",
]


@dataclass(frozen=True)
class SourceReference:
    company_key: str
    source_system: SourceSystem
    source_model: str
    source_id: str


@dataclass(frozen=True)
class AccountingLine:
    account_key: str
    description: str
    debit: Decimal
    credit: Decimal
    tax_keys: tuple[str, ...]


@dataclass(frozen=True)
class AccountingDocument:
    source: SourceReference
    document_type: DocumentType
    partner_key: str | None
    currency: str
    document_date: date
    posting_date: date
    source_updated_at: datetime
    original_reference: str | None
    lines: tuple[AccountingLine, ...]

There are several deliberate choices here:

• Monetary values use Decimal, not float
• Dates and posting dates are separate
• The legal company is part of the source identity
• The original source reference is retained
• Tax and account relationships use controlled mapping keys
• Version-specific Odoo fields do not leak into the target writer

The canonical model acts as an anti-corruption layer between the source applications and Odoo Enterprise.

Build the TallyPrime migration as a staged pipeline

The historical accounting periods extended back to January 2012 for one company and January 2013 for the other.

The restored TallyPrime backups were exported primarily as XML.

We treated the historical migration as a staged pipeline:

Extract
  -> Sanitize
  -> Parse
  -> Classify
  -> Normalize
  -> Map
  -> Validate
  -> Dry run
  -> Commit
  -> Reconcile

Each stage produced output that could be inspected independently.

Extraction

The source scope included:

• Ledgers and chart of accounts
• Customers and vendors
• Sales and purchase vouchers
• Credit and debit notes
• Receipts and payments
• Journal and contra entries
• VAT information
• Bill references
• Bank and cash records
• Relevant stock and inventory reports

Sanitization

Historical XML was not assumed to be perfectly parseable.

The sanitization layer handled issues such as:

• Invalid XML control characters
• Malformed numeric values
• Unexpected encoding
• Missing references
• Empty identifiers
• Inconsistent date formats

For a large export, a streaming parser is preferable to loading the entire document into memory.

from collections.abc import Iterator
from pathlib import Path
from xml.etree.ElementTree import iterparse


def iter_vouchers(xml_path: Path) -> Iterator[dict]:
    for event, element in iterparse(xml_path, events=("end",)):
        if element.tag != "VOUCHER":
            continue

        yield parse_voucher(element)
        element.clear()

The actual parsing rules depend heavily on the Tally export structure and business configuration. The important point is that extraction and interpretation remained separate from target creation.

Classification

Not every source voucher was automatically eligible.

Records were classified as:

• Eligible
• Excluded
• Exceptional
• Cancelled
• Future-dated
• Incomplete
• Requiring manual accounting review

This classification happened before the Enterprise writer was allowed to create anything.

from enum import Enum


class Eligibility(str, Enum):
    ELIGIBLE = "eligible"
    EXCLUDED = "excluded"
    EXCEPTION = "exception"


def classify(voucher: dict) -> tuple[Eligibility, str]:
    if voucher.get("cancelled"):
        return Eligibility.EXCLUDED, "cancelled_voucher"

    if voucher.get("date") is None:
        return Eligibility.EXCEPTION, "missing_voucher_date"

    if not voucher.get("ledger_entries"):
        return Eligibility.EXCEPTION, "missing_ledger_entries"

    if voucher.get("is_future_dated"):
        return Eligibility.EXCLUDED, "future_dated"

    return Eligibility.ELIGIBLE, "ready"

The reason was always retained.

An excluded record must not disappear into a generic "not imported" count.

Use stable source identities

A document's display number is not always a reliable integration key.

Invoice numbers may be:

• Reused between companies
• Reformatted
• Missing from older records
• Changed during corrections
• Duplicated across different document types

We used a composite source identity.

Conceptually, it looked like this:

def source_key(ref: SourceReference) -> str:
    return ":".join(
        [
            ref.company_key,
            ref.source_system,
            ref.source_model,
            ref.source_id,
        ]
    )

Example:

trading:odoo15:account.move:25209

The identity contains:

• Legal company
• Source system
• Source model
• Source record identifier

For TallyPrime records, the original GUID or another stable voucher identifier was retained wherever possible.

Display references remained useful for accountants, but the integration did not depend on display text alone to determine identity.

Design for at-least-once delivery

It is tempting to describe a synchronization service as "exactly once."

Across network boundaries, that claim is usually misleading.

The listener can experience:

• Timeouts
• Process termination
• Network interruption
• API errors
• Partial success
• A restart before the checkpoint is persisted

A more realistic model is:

Deliver at least once, but make processing idempotent.

Each normalized document produced a deterministic fingerprint.

import hashlib
import json
from dataclasses import asdict
from decimal import Decimal


def json_default(value):
    if isinstance(value, Decimal):
        return str(value)
    return value.isoformat()


def document_fingerprint(document: AccountingDocument) -> str:
    payload = json.dumps(
        asdict(document),
        sort_keys=True,
        separators=(",", ":"),
        default=json_default,
    )
    return hashlib.sha256(payload.encode("utf-8")).hexdigest()

The mapping store retained information such as:

from dataclasses import dataclass
from typing import Literal


MappingState = Literal[
    "discovered",
    "target_created",
    "mapped",
    "verified",
    "failed",
]


@dataclass
class RecordMapping:
    source_key: str
    target_model: str
    target_id: int | None
    fingerprint: str
    state: MappingState
    attempt_count: int
    last_error: str | None

Before creating a target record, the service checked:

1. Whether a verified mapping already existed
2. Whether a target record carried the known source marker
3. Whether the current source fingerprint differed from the processed fingerprint
4. Whether the workflow allowed an update

Handle the create-before-checkpoint crash window

One of the most dangerous failure windows looks like this:

1. The service creates a record in Odoo Enterprise
2. Odoo commits the record
3. The integration process crashes
4. The source-to-target mapping is not saved
5. The source record is processed again
6. A second Enterprise record is created

Checking only the local mapping table does not solve this problem.

The retry path also needs a way to rediscover the target record.

A simplified workflow looks like this:

def sync_document(document: AccountingDocument) -> int:
    key = source_key(document.source)
    fingerprint = document_fingerprint(document)

    with source_lock(key):
        mapping = mappings.find(key)

        if mapping and mapping.state == "verified":
            if mapping.fingerprint == fingerprint:
                return mapping.target_id

            return update_existing_document(
                mapping=mapping,
                document=document,
                fingerprint=fingerprint,
            )

        target = enterprise.find_by_source_reference(
            company_key=document.source.company_key,
            source_system=document.source.source_system,
            source_model=document.source.source_model,
            source_id=document.source.source_id,
        )

        if target:
            mappings.repair(
                source_key=key,
                target_model=target.model,
                target_id=target.id,
                fingerprint=fingerprint,
            )
            verify_target(target.id, document)
            return target.id

        target_id = enterprise.create_document(
            build_enterprise_payload(document)
        )

        mappings.mark_target_created(
            source_key=key,
            target_model="account.move",
            target_id=target_id,
            fingerprint=fingerprint,
        )

        verify_target(target_id, document)
        mappings.mark_verified(key)

        return target_id

The exact source-marker strategy depends on the target model and allowed customizations.

Possible approaches include:

• Approved custom source-reference fields
• External identifiers
• Original voucher or invoice references
• A dedicated integration mapping store
• A combination of target markers and external mappings

The key requirement is recoverability after partial success.

Use restart-safe incremental checkpoints

The production listener runs every 20 seconds.

Polling only by numeric ID is not enough because existing records can be updated.

Polling only by timestamp can also be unsafe when multiple records share the same timestamp or the source clock has limited precision.

A safer cursor includes both the last modification timestamp and the record ID.

from dataclasses import dataclass
from datetime import datetime, timedelta


@dataclass(frozen=True)
class Cursor:
    write_date: datetime
    record_id: int

A conceptual Odoo domain for fetching later records is:

def changed_after(cursor: Cursor):
    return [
        "|",
        ("write_date", ">", cursor.write_date),
        "&",
        ("write_date", "=", cursor.write_date),
        ("id", ">", cursor.record_id),
    ]

In production, we also used the integration's idempotency controls to tolerate an overlap window.

SAFETY_WINDOW = timedelta(seconds=60)


def polling_start(cursor: Cursor) -> datetime:
    return cursor.write_date - SAFETY_WINDOW

The overlap intentionally causes some records to be seen again.

That is acceptable because repeated observation is safer than missing an update, provided processing is idempotent.

The checkpoint advances only after the corresponding record reaches a verified state.

for record in adapter.fetch_changed_invoices(
    changed_after=polling_start(checkpoint),
    limit=500,
):
    document = normalize(record)
    sync_document(document)

    checkpoint_store.advance(
        source=document.source.source_system,
        company=document.source.company_key,
        write_date=document.source_updated_at,
        record_id=int(document.source.source_id),
    )

A failed record is written to an exception queue and does not become an invisible gap.

Keep multi-company mappings isolated

The target Enterprise database contains two companies.

That does not mean mappings can be shared freely between them.

A partner with a similar name may exist in both companies. The same account code can have different meaning or configuration. Journals and taxes are company dependent.

Every mapping lookup therefore includes company context.

@dataclass(frozen=True)
class MappingKey:
    company_key: str
    mapping_type: str
    source_value: str


def resolve_account(
    company_key: str,
    source_ledger: str,
) -> int:
    key = MappingKey(
        company_key=company_key,
        mapping_type="account",
        source_value=normalize_name(source_ledger),
    )

    account_id = mapping_registry.get(key)

    if account_id is None:
        raise UnresolvedMapping(
            f"No account mapping for {key}"
        )

    return account_id

The same rule applies to:

• Partners
• Products
• Accounts
• Journals
• Taxes
• Payment methods
• Document sequences
• Source-to-target record relationships

Company isolation is an accounting invariant, not just an application filter.

Treat reverse synchronization as an allowlisted projection

The primary flow remains:

Odoo Community -> Odoo Enterprise Accounting

Selected information also needs to return from Enterprise, including payment-status updates and approved draft-update scenarios.

We did not implement unrestricted bidirectional replication.

Instead, reverse synchronization was treated as an allowlisted projection.

REVERSE_SYNC_RULES = {
    "customer_invoice": {
        "payment_status",
    },
    "vendor_bill": {
        "payment_status",
    },
    "draft_invoice": {
        "approved_reference_fields",
    },
}

Every reverse update must answer:

• Is this document type allowed?
• Is this field allowed?
• Which system owns the field?
• Can the source record still be updated?
• Has this exact target state already been applied?
• Could the update trigger another forward-sync loop?

def apply_reverse_update(event):
    allowed_fields = REVERSE_SYNC_RULES.get(
        event.document_type,
        set(),
    )

    if event.field_name not in allowed_fields:
        raise ReverseSyncNotAllowed(
            f"{event.document_type}.{event.field_name}"
        )

    mapping = mappings.find_by_target(
        target_model=event.target_model,
        target_id=event.target_id,
    )

    if mapping is None:
        raise MissingSourceMapping(event.target_id)

    adapter = source_adapters[mapping.source_system]

    adapter.apply_reverse_projection(
        source_id=mapping.source_id,
        field_name=event.field_name,
        value=event.value,
        correlation_id=event.correlation_id,
    )

A correlation identifier or an equivalent state marker helps prevent the reverse update from being interpreted as a new forward change.

The important rule is that "two-way" must never mean "both systems can update everything."

Validate accounting at multiple levels

An API returning HTTP 200 does not prove that an accounting migration is correct.

We validated at four different levels.

1. Structural validation

• Required fields exist
• Dates can be parsed
• Amounts use valid decimal values
• Documents contain valid lines
• Source references are unique
• Mappings resolve inside the correct company

def validate_structure(document: AccountingDocument) -> None:
    if not document.lines:
        raise ValidationError("document_has_no_lines")

    if document.document_date > document.posting_date:
        raise ValidationError("document_date_after_posting_date")

    for line in document.lines:
        if line.debit < 0 or line.credit < 0:
            raise ValidationError("negative_debit_or_credit")

        if line.debit and line.credit:
            raise ValidationError("line_has_both_debit_and_credit")

2. Accounting validation

For journal-based documents, debit and credit totals must balance.

def validate_balance(document: AccountingDocument) -> None:
    total_debit = sum(
        (line.debit for line in document.lines),
        start=Decimal("0"),
    )

    total_credit = sum(
        (line.credit for line in document.lines),
        start=Decimal("0"),
    )

    if total_debit != total_credit:
        raise ValidationError(
            f"unbalanced_document:{total_debit}:{total_credit}"
        )

3. Reconciliation validation

The migration compared:

• Discovered voucher counts
• Eligible counts
• Excluded counts
• Exception counts
• Target record counts
• Tally GUIDs
• Original references
• Debit and credit totals
• Ledger-line counts
• Duplicate markers
• Accounting periods

A useful reconciliation invariant is:

assert (
    discovered_count
    == eligible_count
    + excluded_count
    + exception_count
)

Every discovered source record must end in a known classification.

4. Report-level validation

Record-level correctness is still not enough.

The accounting team reviewed:

• Trial Balance
• Profit and Loss
• General Ledger
• VAT behavior
• Invoice and vendor-bill references
• Opening balances
• Closing stock
• Accounting periods

The Odoo Trial Balance was aligned to the required six-column view:

• Initial Debit
• Initial Credit
• Period Debit
• Period Credit
• Ending Debit
• Ending Credit

Report-level validation exposed issues that row counts could not, including date mismatches, missing references and stock-treatment differences.

Make rollback ownership-aware

A broad delete command is not a rollback strategy.

In a live Enterprise database, rollback must distinguish between:

• Records created by the migration
• Records updated by the integration
• Genuine production records
• Records created manually by accountants
• Historical records that existed before migration

We used source references, mapping records, migration markers, dry-run output and checkpoints to establish ownership.

Conceptually:

def rollback_record(mapping: RecordMapping) -> None:
    if mapping.state not in {"target_created", "mapped", "verified"}:
        raise UnsafeRollback("unknown_mapping_state")

    target = enterprise.read(
        model=mapping.target_model,
        record_id=mapping.target_id,
    )

    if not target_matches_source_marker(
        target=target,
        source_key=mapping.source_key,
    ):
        raise UnsafeRollback("target_ownership_not_proven")

    enterprise.revert_or_remove(target)
    mappings.mark_rolled_back(mapping.source_key)

If record ownership could not be proven, the automation did not remove it.

That restriction protected genuine accounting data from migration cleanup routines.

Add observability around business outcomes

A production integration needs more than application logs.

Each processing result should be represented as a business outcome.

Useful statuses include:

• Created
• Updated
• Already synchronized
• Skipped by rule
• Excluded
• Waiting for mapping
• Waiting for target posting
• Reverse update applied
• Validation failed
• API request failed
• Verified

A structured event can look like this:

@dataclass(frozen=True)
class SyncResult:
    source_key: str
    target_id: int | None
    company_key: str
    direction: Literal["forward", "reverse"]
    outcome: str
    attempt: int
    checkpoint: str
    error_code: str | None

Operational metrics should answer:

• How many records were discovered?
• How many were created?
• How many were updated?
• How many were identified as duplicates?
• How many failed?
• How far behind the source is the listener?
• What is the latest verified checkpoint?
• Are reverse-sync jobs succeeding?
• Which mappings remain unresolved?

A job that is technically running but no longer advancing its checkpoint is not healthy.

Why XML-RPC and JSON-2 were both used

The two Community systems were existing Odoo 11 and Odoo 15 installations, so XML-RPC remained the practical external interface for those sources.

The Enterprise target used Odoo's newer JSON-2 API.

This also aligns with Odoo's current API direction. The Odoo 19 documentation identifies External JSON-2 as the replacement for the older external XML-RPC and JSON-RPC endpoints.

The integration hid both protocols behind adapters so transport choices did not leak into the accounting pipeline.

class EnterpriseGateway(Protocol):
    def create_document(self, payload: dict) -> int:
        ...

    def update_document(
        self,
        target_id: int,
        payload: dict,
    ) -> None:
        ...

    def find_by_source_reference(
        self,
        company_key: str,
        source_system: str,
        source_model: str,
        source_id: str,
    ):
        ...

    def read_document(self, target_id: int) -> dict:
        ...

This separation also makes a future transport change less disruptive.

The migration logic depends on the gateway contract, not on a particular HTTP or RPC library.

For the current external API direction, refer to the Odoo 19 external API documentation.

Results and engineering lessons

The completed and accepted implementation produced the following validated scope:

• Trading company: 60,606 source vouchers analyzed and 59,459 eligible vouchers migrated and validated
• Solutions company: 27,949 eligible vouchers and 79,157 eligible ledger lines migrated and validated
• Combined total: 87,408 eligible historical vouchers
• Historical coverage: More than 14 years
• Operational sources: Odoo Community 11 and Odoo Community 15
• Target: One multi-company Odoo Enterprise accounting environment
• Synchronization interval: 20 seconds
• Reverse synchronization: Payment statuses and selected draft updates
• Project status: Completed and accepted

Within the confirmed eligible scope, no accounting records were lost.

The most important engineering lessons were not specific to TallyPrime or Odoo.

1. Migration and synchronization must share identity rules

A historical migration and a production listener cannot maintain separate ideas of record identity.

They need the same:

• Source keys
• Company context
• Mapping rules
• Duplicate controls
• Reference strategy

2. At-least-once delivery is acceptable when processing is idempotent

Trying to prevent every repeated read is fragile.

It is safer to tolerate repeated observation and make repeated processing harmless.

3. The crash window after target creation must be recoverable

A local mapping table is not sufficient if the process can crash after the remote system commits.

The target must also be searchable using a stable source marker or equivalent reference.

4. Multi-company context belongs in every key

Company context should be part of:

• Mapping keys
• Source references
• Checkpoints
• Lookup rules
• Validation
• Logging

It should not be inferred later from the user or current session.

5. Reverse synchronization needs explicit ownership

Do not implement generic bidirectional replication for accounting records.

Define approved projections and make field ownership explicit.

6. Accounting validation must reach the report layer

Record counts can be correct while the Trial Balance or Profit and Loss remains wrong.

Financial reports are part of acceptance testing.

7. Rollback must prove record ownership

If the integration cannot prove that it created or modified a record, it should not delete or revert it automatically.

A practical checklist

If you are designing a similar ERP or accounting integration, verify the following before the first production write.

Identity

• Is the source key stable?
• Does it include company context?
• Can the target be rediscovered after a partial failure?
• Are display references separate from integration identity?

Mapping

• Are account, journal, partner and tax mappings explicit?
• Are mappings isolated per legal company?
• Are unresolved mappings visible?
• Are mapping changes versioned?

Processing

• Is every job safe to retry?
• Is the listener using a restart-safe checkpoint?
• Is there an overlap window for updated records?
• Can repeated processing create duplicates?

Reverse synchronization

• Which system owns each field?
• Are reverse fields allowlisted?
• How are synchronization loops prevented?
• Can the source record still accept the update?

Validation

• Do structural validations run before writes?
• Are debit and credit totals checked?
• Do discovered, eligible, excluded and exceptional counts reconcile?
• Are reports reviewed after migration?

Recovery

• Can migration-created records be identified?
• Can rollback avoid genuine production records?
• Are original exports preserved?
• Are dry-run outputs and diagnostic logs retained?

Operations

• Is processing lag measured?
• Are checkpoints monitored?
• Are failures classified?
• Are exceptions actionable?
• Can a new engineer trace a target record back to its source?

Closing thought

The difficult part of ERP integration is rarely the API call.

The difficult part is preserving meaning and ownership across:

• Different accounting models
• Different software versions
• Different legal companies
• Historical data inconsistencies
• Network failures
• Retries
• Human corrections
• Financial reporting requirements

Once identity, idempotency, company boundaries and validation are designed correctly, the transport layer becomes much easier to manage.

The complete business case study and sanitized project evidence are available in the TallyPrime to Odoo Enterprise migration case study.

Disclosure: This article was prepared with AI-assisted editing and reviewed by the Zestminds engineering team against the completed implementation, migration records and accounting-validation results.