Many internal analytics projects never get past the planning stage because teams assume they need a data lake first. In practice, most start with what already exists: spreadsheets and the databases behind existing applications. The work is to check that data, consolidate it and then build on it.
Here is a short checklist we use to think through that first step.
1. Find the silos and the manual steps
- List every place the same figure is kept (each department's sheet, each app's database).
- Note which reports are rebuilt by hand every week or month.
- Mark where teams react to problems only after they show up in a report.
2. Put validation in the pipeline, not in people's heads
- Write the validation rules down and run them inside the ETL/ELT step.
- Decide how exceptions are handled: rejected, flagged or routed to a person.
- Use trigger-based or event-driven runs where data changes often, and scheduled runs where it does not.
3. Automate only what is frequent, structured and manual
- Data entry, reconciliation and report generation are typical candidates for RPA.
- If a task changes every time, fix the process before automating it.
4. Design access before the first dashboard
- Apply role-based access to dashboards, reports and models from day one.
- Build for the real audience: management MIS, departmental reports, programme monitoring or field-team operational views.
5. Only then add prediction
- Forecasting and predictive models (for example with Scikit-Learn, TensorFlow or PyTorch) work far better once the consolidated data is trusted.
More detail: how we approach data engineering, dashboards and AI automation
Written by the team at Affix Center, an IT services company in Lower Parel, Mumbai.
Top comments (0)