DEV Community: Azom Shahriar

Engineering Challenges in Developing a FinTech Platform

Azom Shahriar — Sun, 07 Jul 2024 11:21:43 +0000

I have spent over 13 years working in FinTech engineering and product development across multiple organizations.
From my experience, let me share a few common challenges in developing and engineering FinTech platforms, such as Digital Wallets, Accounting, Transactions, and Payments.
Here we won't discuss common system challenges like performance, scalability, availability, reliability, and risk & security.

1. Accounting

Maintaining double-entry bookkeeping is crucial. When we add money to an account, another contra account must be deducted. In accounting terms, when we credit one account, we need to debit another account simultaneously. This allows us to trace platform money movement and manage internal reconciliation effectively.
Categorize all accounts into two major classes: Assets and Liabilities. Remember:
When an Asset is Debited, it increases.
When a Liability is Credited, it increases.

2. Atomicity & Transaction
Any operation or group of operations should be atomic in nature. ACID-compliant databases are more suitable for this requirement. For example, if you want to perform a payment, multiple operations are involved:

Deducting the amount from the payer's account & Crediting the amount to the payee's account
Deduct payment fee from payer and increase income account
corresponding VAT transaction
Logging the three transaction details.
Some other state update All five operations must be atomic, meaning either all five operations succeed or all fail. There should be no scenario where operations 1, 2, and 4 succeed and operations 3 & 5 fail. We can achieve atomicity by using RDMS transactions feature.

3. Consistency
All write operations should be consistent. For instance, when we update a balance, it should be immediately reflected across all master nodes. Any subsequent read operation should retrieve the updated balance.

4. Double Spending issue
In digital money systems, double spending means a sender can attempt to send or pay the same funds to multiple recipients simultaneously.

For example:
User A has a $100 balance.
At the same time, User A tries to:
Send $100 to User B
Send $100 to User C

To prevent this issue, we need to implement a proper locking mechanism. This will ensure that once a transaction starts, the balance is locked, preventing any simultaneous transactions and avoiding double-spending.

5. Concurrency in Balance update
After each transaction or accounting event, we need to update the account balance. If two concurrent requests try to update the balance, there is a possibility of an incorrect or "dirty" balance.

For example:
User A has a $100 balance.
User A receives $100 from two accounts simultaneously.
The first request reads the balance as $100 and updates it to $100 + $100 = $200.
At the same time, the second request also reads the balance as $100 and updates it to $200.

This results in an incorrect balance update. We need to handle this concurrency issue to ensure accurate balance updates.

6. DeadLock handling
Using a database lock can lead to deadlock, so we need to handle it properly.

7. Duplicate Request handling and Idempotency
The same transaction request or operation can reach the server multiple times due to:

Double-clicking from the browser
Re-transmitted network packets
Client retry mechanisms

A FinTech platform should be able to handle this issue by implementing a unique request ID and idempotent key for each request.

8. Internal Reconciliation
All money movements on a FinTech platform should be reconcilable almost in real-time. This ensures that any unauthorized alterations by intruders or internal resources are detected. If someone alters data or balances at the data layer, the discrepancy will be immediately reflected in the internal reconciliation report.

9. External Reconciliation
When a FinTech platform communicates with other systems, it must be reconciled with the third-party state. Every event should be traceable using both internal and external reference IDs.

10. Audition & Tracing
All state change events must be auditable. If any event occurs, such as a transaction, balance update, account state update, or any other data change, the following information must be captured:

Creator
Modifier
Inputter/Authorizer (Maker/Checker)
Creation time
Modified time
Complete change history

This ensures thorough tracking and accountability for all actions.

11. Append only nature and data immutability
All events and change history should be append-only in nature. Events such as transactions and balance changes should be immutable. This ensures that once an event is recorded, it cannot be altered or deleted.

12. Celebrity account/wallet problem
This issue can arise when an account receives money from millions of senders (e.g., a biller receiving payments nationwide on the last payment date). Regular merchant account lock won't work here. The FinTech core needs to handle these balance and transaction events differently. Similarly, a merchant wallet might need to send/payout/disburse money to millions of receivers at once.

13. Decimal Problem and rounding
Decimal and rounding issues can occur in amount and balance calculations. These need to be handled properly. In Java, it is recommended to use BigDecimal instead of Double for precise arithmetic operations.

14. Insight reports
It's essential to implement insight reports for maintaining the integrity of a FinTech core accounting platform. Examples include balance sheets, trial balances, cash flow statements, payable and receivable reports, and profit and loss statements. These reports provide critical insights into financial health and performance.

15. More important features
Several common feature-level challenges need to be handled properly in FinTech platforms, such as:

Day-end process and close of business procedures
Adjustment and compensation transactions, reversals, and refunds
Sundry and escrow management
Account state management (PENDING, ACTIVE, FREEZE, HOLD, BLOCKED, CLOSED, DEBIT_BLOCKED, CREDIT_BLOCKED)
Management of holding balances and pending transactions
Implementation of the Inputter/Authorizer (Maker/Checker) concept to ensure that large transaction events involve at least two persons for execution

We believe every tech company, whether a startup or enterprise, needs a digital wallet/account/ledger platform to:
Receive, store, and move money
Manage internal accounting and finance
Support both open and closed-loop wallet/account economies
Model complex transactions, such as payments, fund collections, repayments, and loan disbursements.

**At LooFi, we have addressed all these challenges. **The LooFi Digital Wallet/Account/Transaction Engine can be used in:

FinTech
MFI/MFS
E-Wallet/Digital Wallet/Payment Platform/Virtual Card
BNPL/Lending Platform
WealthTech/Investment/Crowdfunding Platform
Digital/Neo Banking
ERP
Any tech enterprise or startup that wants to receive, store, and move money

If you need a digital wallet/account platform, contact us. Our solution can be used independently or integrated with your existing platform. We offer cost-effective assistance in various models:

B2B API SaaS model
On-Prem Deployment
Co-Development & Consultation

Email: azomshahriar05@gmail.com, lognifintech@gmail.com
WhatsApp: +8801674242986

These are not the only challenges when building a robust FinTech platform. There are many more. If you are facing any specific challenges, please comment and share your experience.

Engineering Challenges in Developing a FinTech Platform

Azom Shahriar — Sun, 07 Jul 2024 11:21:43 +0000

1. Accounting

Deducting the amount from the payer's account & Crediting the amount to the payee's account
Deduct payment fee from payer and increase income account
corresponding VAT transaction
Logging the three transaction details.
Some other state update All five operations must be atomic, meaning either all five operations succeed or all fail. There should be no scenario where operations 1, 2, and 4 succeed and operations 3 & 5 fail. We can achieve atomicity by using RDMS transactions feature.

4. Double Spending issue
In digital money systems, double spending means a sender can attempt to send or pay the same funds to multiple recipients simultaneously.

For example:
User A has a $100 balance.
At the same time, User A tries to:
Send $100 to User B
Send $100 to User C

This results in an incorrect balance update. We need to handle this concurrency issue to ensure accurate balance updates.

6. DeadLock handling
Using a database lock can lead to deadlock, so we need to handle it properly.

7. Duplicate Request handling and Idempotency
The same transaction request or operation can reach the server multiple times due to:

Double-clicking from the browser
Re-transmitted network packets
Client retry mechanisms

A FinTech platform should be able to handle this issue by implementing a unique request ID and idempotent key for each request.

Creator
Modifier
Inputter/Authorizer (Maker/Checker)
Creation time
Modified time
Complete change history

This ensures thorough tracking and accountability for all actions.

15. More important features
Several common feature-level challenges need to be handled properly in FinTech platforms, such as:

Day-end process and close of business procedures
Adjustment and compensation transactions, reversals, and refunds
Sundry and escrow management
Account state management (PENDING, ACTIVE, FREEZE, HOLD, BLOCKED, CLOSED, DEBIT_BLOCKED, CREDIT_BLOCKED)
Management of holding balances and pending transactions
Implementation of the Inputter/Authorizer (Maker/Checker) concept to ensure that large transaction events involve at least two persons for execution

**At LooFi, we have addressed all these challenges. **The LooFi Digital Wallet/Account/Transaction Engine can be used in:

FinTech
MFI/MFS
E-Wallet/Digital Wallet/Payment Platform/Virtual Card
BNPL/Lending Platform
WealthTech/Investment/Crowdfunding Platform
Digital/Neo Banking
ERP
Any tech enterprise or startup that wants to receive, store, and move money

If you need a digital wallet/account platform, contact us. Our solution can be used independently or integrated with your existing platform. We offer cost-effective assistance in various models:

B2B API SaaS model
On-Prem Deployment
Co-Development & Consultation

Email: azomshahriar05@gmail.com, lognifintech@gmail.com
WhatsApp: +8801674242986

These are not the only challenges when building a robust FinTech platform. There are many more. If you are facing any specific challenges, please comment and share your experience.

Software Testing guideline for non CS background

Azom Shahriar — Tue, 27 Feb 2024 06:31:19 +0000

Software Testing guideline for non CS background

সফটওয়্যার টেস্টিং সম্পর্কে পরিচিতি:
সফটওয়্যার টেস্টিং হলো একটি গুরুত্বপূর্ণ পদক্ষেপ যা সফটওয়্যার প্রোডাক্টের গুণগত স্থিতি এবং কার্যকরতা নিরীক্ষণ এবং যাচাই করে। এটি সফটওয়্যার উন্নতির জন্য অত্যন্ত গুরুত্বপূর্ণ একটি পদক্ষেপ যা ব্যবহারকারীদের কৌশলের নিরাপত্তা ও সফটওয়্যারের দায়িত্ব নিশ্চিত করে।
একটি বাসার নির্মাণের উদাহরণ চিন্তা করা যায়। ধরুন আপনি একটি নতুন বাসা কিনেছেন এবং বাসা নির্মাণের শুরুতে বিভিন্ন নির্দিষ্ট পদক্ষেপের মাধ্যমে তা পরীক্ষা করতে চান। প্রথমে, ভাবুন আপনি নির্মাণকাজ শুরু করেন। এখানে প্রথম ধাপ হলো নির্মাণকাজের স্থানের সঠিকতা যাচাই করা, যেটি একটি বাসা নির্মাণের প্রধান ধাপ। পরবর্তীতে, নির্মাণকাজ শুরু হলে, সেখানে মূল্যবান পদক্ষেপ হলো প্রতিটি ধাপের উপর পরিমাপ এবং নির্দিষ্ট কাজের সঠিক কার্যকরতা। উদাহরণস্বরূপ, দেওয়াল নির্মাণের জন্য সঠিক উপাদান ব্যবহার করা এবং সঠিক পরিমাপের সাথে তা নির্মাণ করা হয়েছে কিনা এমন। এই পদক্ষেপগুলির সমন্বয়ে, আপনি একটি সুরক্ষিত এবং কার্যকরী বাসা পেতেন।
এই উদাহরণটির মাধ্যমে, আপনি সফটওয়্যার টেস্টিং কীভাবে একটি সফটওয়্যার প্রোডাক্টের গুণগত স্থিতি এবং কার্যকরতা যাচাই করে তা বোঝতে পারেন। যেহেতু এটি একটি পরিপ্রেক্ষিত প্রক্রিয়া, আপনি আপনার প্রোডাক্ট পরীক্ষা করে তা বাড়তি নিরাপত্তা ও সন্তুষ্টিসহকারে ব্যবহারকারীদের কাছে প্রদর্শন করতে পারেন।
উদাহরণ: চিন্তা করা যাক ফেসবুক মোবাইল অ্যাপের টেস্টিং এর একটি উদাহরণ।
ঘটনা: ব্যবহারকারী লগইন
টেস্ট কেস: যাচাই করুন যে ব্যবহারকারীরা ফেসবুক মোবাইল অ্যাপে সফলভাবে লগইন করতে পারেন।
পদক্ষেপসমূহ:
অ্যাপ চালু করুন: ডিভাইসে ফেসবুক মোবাইল অ্যাপ খুলুন।
শংকের তথ্য প্রদান করুন: সঠিক ব্যবহারকারীর নাম (ইমেইল বা ফোন নম্বর) এবং পাসওয়ার্ড প্রদান করুন।
লগইন বোতামে ট্যাপ করুন: "লগইন" বোতামে ট্যাপ করুন।
সফল লগইন যাচাই করুন:

যাচাই করুন যে লগইনের পরে ব্যবহারকারীকে হোম স্ক্রিন/ড্যাশবোর্ডে পুনর্নির্দেশ করা হয়।
যাচাই করুন যে ব্যবহারকারীর প্রোফাইল ছবি বা নাম লগইনের পরে ড্যাশবোর্ডে প্রদর্শিত হয়, যা সফল লগইন প্রতিস্থাপন করে।

ত্রুটি বিষদ যাচাই করুন:

ভুল তথ্য প্রদান করার সময় অ্যাপের ব্যবহারকারীর আচরণ পরীক্ষা করুন।
ভুল তথ্য প্রদানের জন্য যেসব উপযুক্ত ত্রুটি বার্তা প্রদর্শিত হচ্ছে তা যাচাই করুন।
ভুল তথ্য প্রদানের পরে ব্যবহারকারী লগইন না থাকা নিশ্চিত করুন।

মনে রাখুন "আমাকে মনে রাখুন" অপশন:

"আমাকে মনে রাখুন" বা "লগইনে থাকুন" অপশন কার্যকর কি না সেটি পরীক্ষা করুন।
আবার অ্যাপটি বন্ধ করে পুনরায় খোললে ব্যবহারকারী লগইনে থাকে কি না তা যাচাই করুন (যদি প্রযোজ্য হয়)।

লগআউট কার্যক্রম পরীক্ষা করুন:

অ্যাপ থেকে লগআউট করুন।
লগআউট করার পরে লগইন স্ক্রিনে পুনরায় প্রেরিত করা হয় কিনা যাচাই করুন।
লগআউট করার পরে ব্যবহারকারীর সেশন শেষ হয়েছে এবং তাদেরকে আবার লগইন করতে বলা হচ্ছে তা নিশ্চিত করুন।

প্রত্যাশিত ফলাফলসমূহ:
ব্যবহারকারীরা বৈধ তথ্য প্রদান করে সফলভাবে লগইন করতে পারবেন।
ভুল তথ্য প্রদানের জন্য যথাযথ ত্রুটি বার্তা প্রদর্শিত হবে।
"আমাকে মনে রাখুন" অপশন ব্যবহারকারীর লগইন সেশনটি রক্ষা করবে।
লগআউট কার্যক্রম ব্যবহারকারীর সেশন শেষ করে তাদেরকে লগইন স্ক্রিনে পুনরায় প্রেরিত করবে।

উদাহরণ: মনে করা যাক টেস্টিং সময়ে, বৈধ তথ্য প্রবেশ করার পরে "লগইন" বোতামে ট্যাপ করার পরে, ব্যবহারকারীকে হোম স্ক্রিনের বদলে একটি ফাঁকা স্ক্রিনে পুনঃনির্দেশ করা হয়। এটি লগইন প্রক্রিয়ার সঙ্গে সম্পর্কিত একটি সমস্যা নিশ্চিত করে এবং মুক্তি প্রকাশের আগে এই সমস্যা নিষ্ক্রিয় করার জন্য অতিরিক্ত অনুসন্ধান এবং বাগ রিপোর্টিং প্রয়োজন।
এই উদাহরণে, আপনি দেখাচ্ছেন কীভাবে টেস্টাররা সিস্টেমের বিভিন্ন অংশে পরীক্ষা করে তা নিশ্চিত করে যে সফটওয়্যার ব্যবহারকারীদের জন্য সম্পূর্ণ অভিজ্ঞতা এবং সহজতা সরবরাহ করে।
Let's consider an example of testing the Facebook mobile app:
Scenario: User Login
Test Case: Verify that users can log in to the Facebook mobile app successfully.
Steps:
Launch App: Open the Facebook mobile app on the device.
Enter Credentials: Enter the valid username (email or phone number) and password in the respective fields.
Tap Login Button: Tap on the "Login" button.
Verify Successful Login:

Ensure that the user is redirected to the home screen/dashboard after successful login.
Verify that the user's profile picture or name is displayed on the dashboard, confirming a successful login.

Check Error Handling:

Test the app's behavior when invalid credentials are entered.
Verify that appropriate error messages are displayed for incorrect credentials.
Ensure that the user is not logged in with invalid credentials.

Check Remember Me Option:

Test the "Remember Me" or "Stay Logged In" option functionality.
Verify that the user remains logged in even after closing and reopening the app (if applicable).

Test Logout Functionality:

Logout from the app.
Verify that the user is redirected to the login screen after logging out.
Ensure that the user's session is terminated, and they are required to log in again.

Expected Results:
The user should be able to log in successfully with valid credentials.
Appropriate error messages should be displayed for invalid credentials.
The "Remember Me" option should retain the user's login session.
Logout functionality should terminate the user's session and redirect them to the login screen.

Example: Suppose during testing, after entering valid credentials and tapping the login button, the user is redirected to a blank screen instead of the home screen. This indicates a potential issue with the login process, and further investigation and bug reporting are required to address the issue before release.
This example demonstrates how testers can systematically test the Facebook mobile app's login functionality to ensure a smooth user experience.

একজন কম্পিউটার সায়েন্স ব্যাকগ্রাউন্ড ছাড়াও সফটওয়্যার ম্যানুয়াল টেস্টার হতে চাইলে আপনি নিম্নলিখিত ধাপগুলি অনুসরণ করতে পারেন:
সফটওয়্যার টেস্টিং সাধারণ ধারণা বুঝুন:

সফটওয়্যার টেস্টিংর মৌলিক ধারণা সম্পর্কে জানা শুরু করুন। এটি এমন বিভিন্ন ধরণের টেস্টিং (যেমনঃ ফাংশনাল টেস্টিং, রিগ্রেশন টেস্টিং, ইউজাবিলিটি টেস্টিং) এবং টেস্টিং পদ্ধতিগুলির (যেমনঃ ওয়াটারফল, এজাইল) সম্পর্কে জানা যেতে পারে।

সফটওয়্যার ডেভেলপমেন্ট লাইফ সাইকেল (এসডিএলসি) সম্পর্কে জানুন:

সফটওয়্যার ডেভেলপমেন্ট প্রসেস এবং এর বিভিন্ন পর্যায়ের সম্পর্কে জ্ঞান অর্জন করুন, যেমনঃ প্রয়োজনীয়তা সংগ্রহ, ডিজাইন, ডেভেলপমেন্ট, টেস্টিং, এবং ডিপ্লয়মেন্ট। এসডিএলসি সম্পর্কে জানা আপনাকে আপনার টেস্টিং কর্মক্ষমতা প্রায়শই সাহায্য করবে।

বেসিক কম্পিউটার দক্ষতা অর্জন করুন:

যখন আপনার কম্পিউটার সায়েন্স ব্যাকগ্রাউন্ড নেই, তখনও বেসিক কম্পিউটার দক্ষতা অত্যন্ত গুরুত্বপূর্ণ। নিশ্চিত হউন যে আপনি কম্পিউটার ব্যবহার করতে, অপারেটিং সিস্টেম নেভিগেট করতে (উইন্ডোজ, লিনাক্স ইত্যাদি) এবং সাধারণ সফ্টওয়্যার অ্যাপ্লিকেশন ব্যবহার করতে সামর্থ্যশালী হয়েছেন।

টেস্টিং টুল এবং প্রযুক্তি সম্পর্কে জানুন:

প্রসিদ্ধ টেস্টিং টুল এবং প্রযুক্তি সম্পর্কে জ্ঞান অর্জন করুন যা উদাহরণস্বরূপ বাগ ট্র্যাকিং টুল (যেমনঃ জিরা, বাগজিলা), টেস্ট ম্যানেজমেন্ট টুল (যেমনঃ টেস্টরেল, জেফার) এবং সহযোগিতা প্ল্যাটফর্ম .

একটি ৩ মাসের কোর্স আউটলাইন নিম্নলিখিত হতে পারে:
মাস ১:
সফটওয়্যার টেস্টিং পরিচিতি:
সফটওয়্যার টেস্টিং এর মৌলিক ধারণা
টেস্টিং প্রক্রিয়াজাতকরণ এবং উপকারিতা
টেস্টিং প্রক্রিয়ার প্রধান পদ্ধতি (উদাহরণস্বরূপ: পার্সিং, ইন্টিগ্রেশন, রিগ্রেশন)

টেস্টিং টুলস এবং প্রযুক্তি:
বাগ ট্র্যাকিং টুল (উদাহরণস্বরূপ: জিরা)
টেস্ট ম্যানেজমেন্ট টুল (উদাহরণস্বরূপ: টেস্টরেল)
সহযোগিতা প্ল্যাটফর্ম (উদাহরণস্বরূপ: স্ল্যাক)

প্রাক্টিক্যাল এপ্লিকেশন:
ওয়েবসাইট টেস্টিং
মোবাইল অ্যাপ্লিকেশন টেস্টিং
বাগ রিপোর্ট তৈরি

মাস ২:
প্রফেশনাল টেস্টিং পদ্ধতি:
প্রফেশনাল টেস্টিং প্রক্রিয়া
প্রফেশনাল টেস্টিং সংশ্লিষ্ট নিয়ম
বাগ রিপোর্টিং এবং ট্র্যাকিং

অটোমেশন টেস্টিং পরিচিতি:
অটোমেশন টেস্টিং কি?
অটোমেশন টেস্টিং প্রসেস

অটোমেশন টুলস:
সেলেনিয়াম
এটিয়াস
পাইটেস্ট

মাস ৩:
এডভান্সড টেস্টিং পদ্ধতি:
ইন্টিগ্রেশন টেস্টিং
সিস্টেম টেস্টিং
পারফর্মেন্স টেস্টিং

স্পেশালাইজড টেস্টিং:
সিকিউরিটি টেস্টিং
ইউজাবিলিটি টেস্টিং
অ্যাক্সেসিবিলিটি টেস্টিং

প্রকল্প অনুশীলন:
প্রকল্পের টেস্টিং
বাগ ফিক্সিং এবং পরিমার্জন

উপরোক্ত অনুশীলন সাথে যুক্ত হতে পারে যেকোনো মূল্যবান প্রকল্পের অংশগুলির উপরে ভিত্তি করে।
এই কোর্সের প্রতিটি সেশন নিম্নলিখিত রকমের প্রশিক্ষণ এবং গাইডেলাইন প্রদান করা যেতে পারে:
ক্লাসরুম প্রশিক্ষণ
অনলাইন ভিডিও টিউটোরিয়াল
অনলাইন কোর্স মাধ্যমে প্রশিক্ষণ
প্রোজেক্ট ও কেস স্টাডি
রিয়েল

Month 1: Introduction to Software Testing:
Basic concepts of software testing
Testing process and benefits
Main testing methodologies (e.g., parsing, integration, regression)

Testing Tools and Technologies:
Bug tracking tools (e.g., Jira)
Test management tools (e.g., TestRail)
Collaboration platforms (e.g., Slack)

Practical Application:
Website testing
Mobile application testing
Bug reporting

Reference Links:
Introduction to Software Testing: YouTube - Software Testing Tutorials for Beginners
Testing Tools Tutorial: Bangla - সফটওয়্যার টেস্টিং টিউটোরিয়াল

Month 2: Professional Testing Methodologies:
Professional testing process
Relevant rules of professional testing
Bug reporting and tracking

Introduction to Automation Testing:
What is automation testing?
Automation testing process

Automation Tools:
Selenium
Appium
Pytest

Reference Links:
Professional Testing Methodologies: Indian - Software Testing Tutorial for Beginners
Automation Testing Tutorial: Bangla - অটোমেশন টেস্টিং টিউটোরিয়াল

Month 3: Advanced Testing Methodologies:
Integration testing
System testing
Performance testing

Specialized Testing:
Security testing
Usability testing
Accessibility testing

Project Implementation:
Testing in projects
Bug fixing and regression

Reference Links:
Advanced Testing Methodologies: Indian - Advanced Software Testing Tutorial
Specialized Testing Tutorial: Bangla - বিশেষজ্ঞ টেস্টিং টিউটোরিয়াল

Software Testing Outline:
Module 1: Introduction to Software Testing

Understanding the basics of software testing
Importance of software testing in software development lifecycle (SDLC)
Different types of testing (e.g., functional testing, non-functional testing)
Overview of manual testing vs. automated testing
Module 2: Test Planning and Documentation

Test planning process
Creating test plans and test cases
Test case design techniques (e.g., equivalence partitioning, boundary value analysis)
Test data preparation and management
Module 3: Test Execution and Reporting

Test execution process
Techniques for executing test cases
Defect reporting and management
Regression testing and retesting
Module 4: Test Management Tools

Introduction to test management tools (e.g., TestRail, Zephyr)
Using test management tools for test planning, execution, and reporting
Integrating test management tools with bug tracking systems (e.g., Jira)
Module 5: Specialized Testing Techniques

Exploratory testing
Usability testing
Compatibility testing (e.g., cross-browser, cross-device)
Security testing basics
Module 6: Agile Testing Practices

Understanding Agile methodology
Agile testing principles and practices
Role of a tester in Agile teams
Agile testing tools and techniques
Module 7: Real-world Project Experience

Working on a real-world testing project
Test case creation, execution, and reporting
Collaborating with team members and stakeholders
Feedback and improvement process
Module 8: Test Closure and Documentation

Test closure activities
Generating test summary reports
Lessons learned and continuous improvement

These reference links provide additional resources for self-study and further understanding of the topics covered in the course outline.
Please search with this text at Youtube.
সফটওয়্যার টেস্টিং প্রাথমিক বিষয়বস্তু | Software Testing Basic Level
Watch on YouTube
সফটওয়্যার টেস্টিং সম্পর্কে পরিচিতি | Introduction to Software Testing in Bangla
Watch on YouTube
সফটওয়্যার টেস্টিং টিউটোরিয়াল - বাংলা | Software Testing Tutorial Bangla
Watch on YouTube

সফটওয়্যার টেস্টিং পরিচিতি - অনলাইন কোর্স - বাংলা | Software Testing Basics - Online Course in Bangla
Watch on YouTub
সফটওয়্যার টেস্টিং প্রথমিক ভূমিকা | Software Testing Basic Role
Watch on YouTube

Search with this at youtube —

Software Testing Tutorial for Beginners — Quality Assurance Training — Edureka
Here are some Youtube links for basic Software manual testing in bangla:

QA manual testing tutorial for beginners | software testing course in bangla | Introduction | Part-1 by Software Testing Academy
Software Testing Tutorial in Bangla PART 1 | Manual Testing Bangla Tutorial | What is SQA by Software Testing Bangla Tutorial
Manual Software Testing Training Part-1 by SDET- QA
সফটওয়্যার কোয়ালিটি এস্যুরেন্স -প্রথম পর্ব Software Quality Assurance in Bangla — First Episode by Shah Amanat
QA manual testing tutorial for beginners | software testing course in bangla | Introduction | Part-2 by Software Testing Academy
Here are some Youtube links for basic software testing in english:

https://www.youtube.com/playlist?list=PLUDwpEzHYYLseflPNg0bUKfLmAbO2JnE9

Software Testing Tutorial #1 — What is Software Testing | With Examples by Software Testing Mentor
Software Testing Full Course In 10 Hours | Software Testing Tutorial | Edureka by edureka!
Lesson 1 — Software Testing by Skillrill — IT Bootcamp and Recruitment
Software Testing Tutorial For Beginners | Manual & Automation Testing | Selenium Training | Edureka by edureka!
Introduction to Software Testing. || The Essential Guide to Software Testing by Coders Arcade

Code and System Design Review Checklist

Azom Shahriar — Fri, 10 Sep 2021 12:32:28 +0000

When writing code, we need to review our own code and other's code as well as software system design and architecture. In this article, we try to share few important notes regarding code and system review.
This checklist is mostly for java backend development. But can also be applied to other technology stacks.
The checklists/notes will help developers ensure better code and system architecture.

Category/Area of Review:

General
Clean Code & Code style
Security
Performance
Logging and Tracing
Concurrency
Error Handling
Maintainability & Testability
Domain(Business)
Architecture
Scalability
Reliability & Resiliency
Design pattern
PCI DSS(FinTech)
REST API design

General:
Use checked exceptions for recoverable conditions and runtime exceptions for programming errors
Try to use global exception handling and handle common Business and technical error response
Never ignore exceptions. Don't overlook the catch block.
Return empty arrays or collections, not nulls.
Minimize the scope of local variables for earlier GC.
Always override hashcode when overriding equal.
Always override toString
Use marker interface to define the type
Use an executor thread pool for tasks and thread instead of unlimited thread creation.
Use the BigDecimal valueof method for the string to bigdecimal/double conversion, otherwise, you will face a precision issue.
Try to avoid string literals at business logic check. Use enum or constant for maintainable code.

Bad:
if(status = "SUCCESS")
Good:
if(status = EventStatus.SUCCESS)
Throw Exceptions rather than Return codes in case of business and technical error.

Bad:
private String checkInput(Request request){
if(something wrong)
return "FAILED";
}
Good:
private String checkInput(Request request){
if(something wrong)
throw new BusinessErrorException(int code, String message)
}

Clean Code:
Use Intention-Revealing Names for variable

Bad:
void validate(String input)
Good:
void validateCardNumber(String cardNumber)
Pick one word per concept.
Use Solution/Problem Domain Names
Classes should be small!
Functions should be small!
Do one thing in a function.
Don't Repeat Yourself (Avoid code Duplication).
Explain yourself in code(write why in code not what)
Make sure the code formatting is applied(Can use tools)
Each method should do a single task. Don't mix business logic and network calls with the same method. Try to make the method unit testable.

Bad:
public sendSms(){
// code for validation
if(mobile is valid mobile)
// code for sms generation
String smsBody = "Some Text"+variable+"text".
// network call to telco server
restTemplate.exchante(url,mobile,smsBody);
}
Good:
public processSms(){
validateMobile();
String smsBody = generateSmsBody(inputs);
sendSms(inputs)
}
private void validateMobile(String mobileNo){}
private String generateSmsBody(Inputs….){}
private sendSms(){}

Security:
Check access control or authorization besides authentication.

Bad:
Only verify the JWT token
Good:
Verify token and check use have authority to access that resource.
All change event applications should be auditable(who performed this operation from which device and IP)

Bad:
account(name,status,createdDate)
Good:
account(name,status,createDate,createdBy,lastModifiedDate,lastModifiedBy)
Use password as an array of characters instead of String so that no one can get it from heap dump.
Make class final if not being used for inheritance.
Input into a system should be checked for valid data size and range and check mandatory input fields(boundary conditions)

Bad:
TxnRequest{
String fromAc;
String toAc;
BigDecimal amount;
}
Good:
TxnRequest{
@notblank(message="From AC Can't be blank.")
String fromAc;
@notblank(message="To AC can't be blank.")
String toAc;
@notnull(message="Amount can't be null.")
@positive(message="Amount should be positive")
BigDecimal amount;
}
Avoid sensitive data logging(like pin, password, card info)

Bad:
log.info("User:{}, Password{}",request.getUserName(),request.getPassword());
Good:
log.info("UserName:{}, Password:****",request.getUserName())
Purge sensitive information from exceptions (exposing file path, internal system configuration)
Be careful about SQL injection when DB queries.
Check the API response fields. Is there any extra data or sensitive data are shared with the public?

Bad:
Ac Balance response:
{"account":"10421020025347",
"balance":550.00,
"modifiedBy":"bank-checker-001",
"userId":456,
"customerInfo":{….}}
Good:
{"account":"10421020025347","balance":550.00}
Define wrappers around native methods (not declare a native method public).
Make public static fields final (to avoid caller changing the value)
Try inter-service communication in a secured way(Implement SSL when service to service call)
Separate public and private/internal API paths so that DevOps team can implement infra-level security and filter.

Bad:
api/v1/all public/admin/internal path,
Good:
api/v1/public path
api-admin/v1/admin related path
api-internal/v1/interapi
In the case of microservice try to use a central auth server
Check about authentication and authorization network call overhead in the case of a distributed system.
Never use default credentials at production. Especially for system/infrastructure-related services. (DB, Cache, Auth, API GW, HTTP server, 3rd party library)
Encrypt or one-way hash for OTP and other credentials.
Ensure REST API security.

Performance:
Try to keep synchronized section small operation(CPU/network/memory)
Avoid string literal concatenation back end component. Try to use a string builder.

Bad:
String fullMessage = "";
for(Result result : resultList){
fullMessage = fullMessage+result.getMessagr();
}
Good:
StringBuilder fullMessage = new StringBuilder();
for(Result result : resultList){
fullMessage.append(result.getMessage()));
}
Avoid creating unnecessary objects.
In case of network call use a connection pool, thread pool, socket pool.
Profile DB query and check high search/read query happen on an indexed field.
Release resources (Streams, Connections, etc) in all cases.
Careful about ORM N+1 query and use Entitygraph to avoid N+1 query
Always think about cache.
In a distributed system(SOA or Microservice) always develop stateless service.
Try to develop as many Asynchronous processes using JMS.
Think about IO latency and CPU usage. If the system is an IO incentive try to use the NIO library.
Check if any unused library goes into the production build.
In the case of an in-memory store avoid full object serialization and use custom serialization to save memory. Think about JSON vs Messagepack vs protobuf.
Try to avoid select * if not needed at DB query.
Try always index-only scan, select the field which is indexed. The most important and readable field should be indexed. We can also use composite indexed-key.

Logging & Tracing:
Maintain proper log level
Use placeholder (LOG.error("Could not… details: {}", something, exception)) and never String concatenation (LOG.error("Could not… details: " + something, exception))
Don't trace excessive logs.
In the case of Distributed Systems (SOA, Microservice) try to use Distributed tracing (Spring cloud sleuth, ELK, Zipkin)
Don't log Sensitive Information
Log User Context
In the case of centralized logs use common trace id and service name.

Concurrency:
Avoid excessive synchronization for thread safety. Try to avoid sharing resources in case of a multithreaded environment.
Avoid member variables when in the case of a singleton object or bean.
Always synchronize share resources and also try to avoid share resource
Use concurrent hashmap instead of Synchronize HashMap
Use HashMap or HashSet instead if TreeMap/Set when ordering is not important. As time complexity O(1) vs O(logn)
Always think about concurrency in the case of back-end service.
When multiple users/threads update the same data, try to implement a lock(optimistic or pessimistic). For example account balance, ticket reservation, product stock.

Error Handling:
Reply consistent error response to the client.
Handle proper error code(401,404,400 and 500)
Use custom error code for the business logic errors.

Typical Error Categories

Business logic error
Technical Error
Upstream Service Error
Common Error/Runtime Error Architecture: Follow twelve-factor app(https://12factor.net) Aware about distributed system fallacy (https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing) For the data layer we can follow lambda architecture. Try to implement event-based infrastructure for scalability. Follow 12 architecture principle

1.N+1 Design. Never less than two of anything and remember the rule of three
2.Design for Rollback. Ensure you can roll back any release of functionality
3.Design to be disabled. Be able to turn off anything you released
4.Design to be monitored. Think about monitoring during design. Not after.
5.Design for multiple Live sites. Don't box yourself into one site solution
6.Use mature technology. Use thing you know work well
7.Asynchronous design. Communicate synchronously only when absolutely necessary
8.Stateless Design. Use state only when the business return justifies it .
9.Scale Out NOT Up. Never rely on a bigger or faster system.
10.Design for at least two axes. Think one step ahead of your scale beads.
11.Bue when Non Core. If you aren't the best at building it and it doesn't offer competitive differentiation, buy it.
12.Commodity Hardware. Cheaper is better most of the time.
Design Pattern:
SOLID
DRY
KISS
Creational Pattern(Singleton,Factory,Builder,Adapter)
Behavioral Pattern(Strategy, Chain Responsibility, Observer).

Scalability:
Make the system elastic or easily scalable.
Make the service stateless for horizontal scaling.
DB Sharding
DB Partition
DB replication
Use read replica for an independent read operation.
Cache high read and low write operation.
Monitor DB read and write query ratio.
Reduce network call overhead (use socket pool or gRPC).
Check DB network call round trip and always try to reduce it.
Address the c10K problem for network communication.

Reliability & Resiliency:
Handle Timeout in case of all network call.
Implement Circuit Breaker Pattern.
Implement Bulkhead pattern
Implement idempotent operation
Use a fail-first mechanism when the system is really broken.

Maintainability & Testability:
How much test coverage does your system have?
Unit, Integration, and System Testing coverage.
Proper CI/CD pipeline.
Do as much automation.
Implement proper production deployment test strategy( canary, AB testing)

Monitoring/Observability:
Monitor your system's health and resources.
Database Memory and disk size
Application and DB server CPU, memory and Load average, and IO
Application latency and throughput.
Application resource(JVM memory, thread, thread pool, connection pool, queue, JMS memory)
Isolate Logging, Tracing, and metric analysis.
Monitor your business performance.

Business Domain:
Business Logic implemented properly at the codebase
Service properly isolated against the domain and bounded context.
Try to follow the domain-driven design.

PCI DSS:(For FinTech):
12 requirements:
1.Install and maintain a firewall configuration to protect cardholder data
2.Do not use vendor-supplied defaults for system passwords and other security parameters
3.Protect stored cardholder data
4.Encrypt transmission of cardholder data across open, public networks
5.Use and regularly update anti-virus software or programs
6.Develop and maintain secure systems and applications
7.Restrict access to cardholder data by business need to know
8.Assign a unique ID to each person with computer access
9.Restrict physical access to cardholder data
10.Track and monitor all access to network resources and cardholder data
11.Regularly test security systems and processes
12.Maintain a policy that addresses information security for all personnel
REST API Design:
Use kebab-case for URLs

Bad: /systemOrders or /system_orders
Good: /system-orders
Use camelCase for Parameters

Bad: /system-orders/{order_id} or /system-orders/{OrderId}
Good: /system-orders/{orderId}
Plural Name to Point to a Collection

Bad: GET /user or GET /User
Good: GET /users
Keep Verbs out of Your Resource URL

Bad: POST /updateuser/{userId} or GET /getusers
Good: PUT /user/{userId}
Use Verbs for Non-Resource URL or specific operation

POST /alerts/245743/resend
Use camelCase for JSON property

Bad:
{
user_name: "Mr Rahim"
user_id: "1"
}
Good:
{
userName: "Mr. Karim"
userId: "1"
}
Don't use table_name for the resource name

Bad: product_order
Good: product-orders
This is because exposing the underlying architecture is not your purpose.
Use API version

Good: http://api.domain.com/v1/shops/3/products
Accept limit and offset Parameters

GET /shops?offset=5&limit=5
Don't Pass Authentication Tokens in URL

Bad:
GET /shops/123?token=some_kind_of_authenticaiton_token
Good:
Instead, pass them with the header:
Authorization: Bearer xxxxxx, Extra yyyyy
Use the Relation in the URL For Nested Resources

GET /shops/2/products : Get the list of all products from shop 2.
GET /shops/2/products/31: Get the details of product 31, which belongs to shop 2.
CORS

Do support CORS (Cross-Origin Resource Sharing) headers for all public-facing APIs.
Consider supporting a CORS allowed origin of "*", and enforcing authorization through valid OAuth tokens.
Avoid combining user credentials with origin validation.
Security
Enforce HTTPS (TLS-encrypted) across all endpoints, resources, and services.
Enforce and require HTTPS for all callback URLs, push notification endpoints, and webhooks.
Reviewer & reviewee's behavior and attitude when reviewing other's code:
Be kind
Accept that many programming decisions are opinions. Discuss trade offs, which you prefer, and reach a resolution quickly.
Ask questions; don't make demands. ("What do you think about naming this :user_id?")
Ask for clarification. ("I didn't understand. Can you clarify?")
Avoid selective ownership of code. ("mine", "not mine", "yours")
Avoid using terms that could be seen as referring to personal traits. ("dumb", "stupid"). Assume everyone is intelligent and well-meaning.
Be explicit. Remember people don't always understand your intentions online.
Be humble. ("I'm not sure - let's look it up.")
Don't use hyperbole. ("always", "never", "endlessly", "nothing")
Be careful about the use of sarcasm. Everything we do is public; what seems like good-natured ribbing to you and a long-time colleague might come off as mean and unwelcoming to a person new to the project.
Consider one-on-one chats or video calls if there are too many "I didn't understand" or "Alternative solution:" comments. Post a follow-up comment summarizing one-on-one discussion.
If you ask a question to a specific person, always start the comment by mentioning them; this ensures they see it if their notification level is set to "mentioned" and other people understand they don't have to respond.

Resources:
https://owasp.org/www-pdf-archive/OWASP_Code_Review_Guide_v2.pdfhttps://dzone.com/articles/java-code-review-checklisthttps://www.codementor.io/blog/code-review-checklist-76q7ovkaqjhttps://owasp.org/www-pdf-archive/OWASP_SCP_Quick_Reference_Guide_v2.pdfhttps://dimikcomputing.com/course/clean-code-online-course/https://stackoverflow.com/questions/7186204/bigdecimal-to-use-new-or-valueofhttps://www.geeksforgeeks.org/use-char-array-string-storing-passwords-java/https://betterprogramming.pub/10-essential-tips-for-writing-secure-rest-api-e297990d48c5https://cloud.google.com/architecture/application-deployment-and-testing-strategieshttps://docs.gitlab.com/ee/development/code_review.html

Sonarqube Installation

Azom Shahriar — Mon, 02 Aug 2021 13:30:47 +0000

SonarQube Installation:

What is SonarQube?
Ans: SonarQube is an open source platform for continuous inspection of code quality & security. Automatic code review with static analysis to detect bug, code smell, security vulnerability around 20+ languages.It also offers reports of duplicate code, coding standard, unit test, code coverage, code complexity, comments, bugs, security vulnerability.

Server Installation using docker:
Default server 9000 port
Can be installed by docker , manual ubuntu installation
Ubuntu Installation (https://developerinsider.co/install-sonarqube-on-ubuntu/)

Docker command:
docker pull sonarqube
docker run -d -p 9000:9000 sonarqube

Integration with your code:

Create Project from sonarqube server interface
Choose project-key and display name
Copy the token from the server.

Changes build.gradle-

plugins {
id "org.sonarqube" version "3.0"
}
Gradle Command for Single Module App:

./gradlew sonarqube \
-Dsonar.projectKey= \
-Dsonar.host.url= \
-Dsonar.login=
Gradle Command for Multi module App:

./gradlew ::sonarqube \
-Dsonar.projectKey=\
-Dsonar.host.url= \
-Dsonar.login=

After running command you will find all vulnerability at sonarqube server dashboard.

5 idea for a Java backend developer can do free time.

Azom Shahriar — Sun, 25 Jul 2021 13:31:31 +0000

1.Involving Open Source community project like (Apache Kafka, Apache pinot, Apache fineract, Apache tomcat, Spring Project, MySQL, PostGRE, TIDB, Redis, Linux, Kubernetes, keycloak.) Or Involve any interesting project/team that can help in more learning.

2.Learn new tech like BlockChain,DeFI,NFT,AI& Data Science.

3.Continue Programming & DSA practice.

4.Educate Community /Junior or create tech content.

5.Continue Learning(Clean Code, Architecture, Design Pattern, SOLID,DRY, System Design,Security,Performance,Scalability,Distributed System, Micro Service,K8, new language like go, rust, elixir)

BackEnd Development checklist to ensure application performance and maintainable code.

Azom Shahriar — Tue, 20 Jul 2021 20:00:27 +0000

Few notes for backend developer which will help to write better production code.

1.Always check any possibility of a null pointer exception.

2.Remove redundant/extra DB or network call. (Query in a loop, N+1 query)
3.In case of database connection, properly use connection pool, pool size, idle size, queue size, monitor pool resource.

4.When querying/searching, think about table size and param index or any performance impact at high load.

5.Measure and profile Database query latency.

6.Handle DB TimeOut(Connection, Read) properly.

7.When network call (REST, in-memory DB, Message Queue), use proper connection pool configuration. (Pool Size, Queue Size, Rejection or exhaustion policy).

8.When third party call, handle Time out(connection & read), try to implement circuit breaker and bulkhead pattern.

9.Always think about the asynchronous process.

10.Be careful about shared resources and critical sections.

11.Try to avoid member variable at spring singleton bean.
Class Name, class size, method name, method size, and variable name should follow clean code rules.

12.Ask yourself, is my method unit testable?

13.Meaningful tag after each release.

14.Every git commit will be an independent feature.

15.Use cache as much as possible.

16.Think about Spring Bean Scope(Singleton, Request & prototype).

17.Try as much automation(Testing, CI & CD)

18.Try to retrieve data from pre-calculated data using the scheduled job.

19.Avoid redundant or any extra variable assignment.

20.Should maintain a proper log with the appropriate level.

21.Follow logging and tracing standard rules.

22.Try as much documentation or note so that QA & System team and another stakeholder do not ask the Developer. Mostly when developers change the config, DB structure, migration script, framework version, library, runtime environment(JDK, python), dependency, new technology.

23.Isolate scheduled jobs from the main application and use standard tools(Jenkins).

24.While writing complex queries, check raw queries and analyze the query using DB tools(like MySQL explain command).