What to do in such situations: Developing an app integrated with generative AI

Hello, I’m pj-corridor, an engineer. In this article, I will introduce the challenges I faced while developing an application integrated with generative AI, such as dealing with abuse and navigating trial-and-error and trade-offs in prompt tuning. I previously had the opportunity to work with Synergy Marketing, and I am grateful that they kindly agreed to publish this article on the TECHSCORE BLOG 😊 Thank you very much.

First, please try the personality assessment app I initially developed. The annoying ads 😆 can be hidden by adjusting the window width.

• CAPS (= Controller, Analyzer, Promoter, Supporter) assessment
• DiSC (= Drive, Influence, Steadiness, Compliance) assessment

Like the popular MBTI, CAPS and DiSC are so-called pseudoscientific personality assessments. They should not be used as a basis for decision-making, but for example, having workshop participants share their results can help energize self-introduction sessions and warm up the atmosphere. Please consider using this assessment app on such occasions.

For reference, here is my CAPS assessment result.

The advice and user manual are generated by AI, but the sudden appearance of “Respondent” in Japanese sentences feels a bit unnatural. In this way, while generative AI provides rich expressive capabilities, it also introduces quality risks. That said, I wanted to keep the cost of ensuring quality within the scope of a hobby project. Therefore, I kept the rule-based scoring logic deterministic and limited the generative AI output to supplementary text, thereby controlling the impact of hallucinations.

From here, I will introduce concrete examples of trial and error and how I approached trade-offs, divided into three chapters.

1. Abuse prevention

The assessment app can be accessed anonymously, but the backend uses a paid generative AI service (Amazon Bedrock). Therefore, being overwhelmed by bot abuse is a plausible scenario. However, rather than aiming to eliminate all threats, I adopted a basic policy of considering the attacker’s incentives: “accept threats with low manifestation risk, and mitigate the rest.”

1.1. Blocking common attack traffic with WAF

For WAF, referring to AWS’s standard protection pack, I adopted:

• GeoRule (blocking IPs from regions with frequent attacks)
• AWS-AWSManagedRulesAmazonIpReputationList (blocking known malicious IPs)
• AWS-AWSManagedRulesAnonymousIpList (blocking anonymized/tunneled IPs)
• A tuned GlobalRateBasedRule for the assessment app (request rate limiting)
• A tuned RateBasedRulePOST (limiting POST/PUT/DELETE requests)

and removed unnecessary rules such as AWS-AWSManagedRulesSQLiRuleSet. However, when using a Lambda Function URL as a public endpoint, there is concern that requests not routed through CloudFront could bypass the WAF. While it is possible to enforce CloudFront (and thus WAF) routing, doing so would require AWS-specific implementations, such as:

• Changing the Lambda Function URL AuthType to AWS_IAM and using CloudFront OAC 👉 This requires support for x-amz-content-sha256 in POST requests
• Adding a secret header in CloudFront and verifying it in Lambda 👉 This requires managing multiple environment variables and verification logic

In any case, AWS-dependent implementation would be necessary. Considering the risk that a malicious actor could discover the Lambda Function URL and bypass the WAF, I decided not to implement this measure for now.

As a side note, settings configured in the AWS console are easy to forget. Forgetting to revert temporary settings can cause issues. To avoid this, I recommend managing managed service configurations in a repository (something close to IaC).

For example, for a WAF protection pack:

1. Manage the JSON as the Source of Truth in the repository
2. Always apply changes via Source of Truth → AWS console
3. Check consistency between Source of Truth and the applied WAF policy via CI

Also, as a follow-up: with the traffic scale of this app, the fixed cost of WAF ended up exceeding the usage-based cost of Bedrock. A good lesson in estimating costs 😅. Going forward, I am considering suspending WAF until traffic increases and instead implementing POST request limits using DynamoDB.

1.2. Controlling access from simple bots and browsers bypassing the flow

For access control, I implemented:

1. When accessing the app, generate a token from “secret information + timestamp” and send it via Set-Cookie
2. The browser includes the token in requests using credentials: 'include'
3. The server validates that the token could have been generated within a human response time window

Additionally, I applied:

• SameSite=Strict to block cookies in cross-domain POST requests (see SameSite explanation)
• Restricting Access-Control-Allow-Origin to a whitelist to limit browser-based access

These measures can be bypassed by more sophisticated bots, but I decided to start within the scope of the basic policy and consider additional measures while monitoring access patterns.

2. Technology selection

Here I describe trial and error related to Lambda and API Gateway.

Lambda was a reasonable choice for the runtime environment, but I initially preferred:

1. Lambda container image (using PHP)
2. Lambda zip deployment (using Python/Node)

I leaned toward (1) because I already had a local PHP testing environment, which seemed suitable for agile development and testing. However, considering compatibility with Lambda and concerns about CI complexity, I ultimately chose (2). In hindsight, I spent more time dealing with AWS environment nuances and black-box behavior, so it turned out to be the right choice.

Speaking of trial and error, repeatedly creating Lambda functions via the AWS console automatically generates new IAM roles each time. Not just these remnants, but leaving unused resources can become technical debt, so they should be cleaned up promptly.

I resisted the urge to jump into development and prepared a mechanism to handle Lambda and test environments transparently.

• Shared Python template
• Test environment launcher

This later became the foundation for prompt tuning.

As for whether to use API Gateway:

Feature	Implementation
Authentication	Custom access control
Routing	Handled within Lambda
Throttling	Handled by WAF (planned to change)

Given this, I decided not to use API Gateway at this stage, as the cost outweighed the benefits.

3. Application development

With abuse prevention and technology selection in place, it was time for development. Once completed, I wanted to share it globally via Reddit. That led to the motivation to support multiple languages, so I implemented a simple i18n class to support 9 languages.

const GREETING = i18n.text({
    en : 'Hello',
    ja : 'こんにちは',
    fr : 'Bonjour',
    de : 'Hallo',
    es : 'Hola',
    pt : 'Olá',
    hi : 'नमस्ते',
    ko : '안녕하세요',
    zh : '你好'
})

To stabilize AI output quality, I used the user’s native language for UI but fixed the AI input language to English.

I also unified state management, UI components, and interfaces with Lambda/test environments between DiSC and CAPS, enabling reuse for future assessments like MBTI.

Finally, I focused on improving and stabilizing AI output quality through prompt tuning. Instead of rushing, I first built a foundation:

1. Prepare shortcut features for prompt generation
   • For example, “random answers + query execution” automation
   • Abuse prevention via hash-based validation
2. Tune in a test environment (without AI connection)
   1. Execute shortcut
   2. Output prompts to console
   3. Simulate production using Gemini/ChatGPT and evaluate
   4. Iterate prompt/template improvements
3. Tune in production (with AI connection)
   1. Partial tuning via shortcut
   2. Full app testing and final checks

During this process, I discovered missing outputs in hi, ko, and zh, likely due to token differences across languages.

Also, AI suggestions tend to bloat prompts with redundant instructions, so consolidating and refactoring prompt structure is recommended.

Finally, about “Respondent”: the requirement was to fix the subject as colleagues/friends while referring to the user consistently. Attempts like:

...use the {{lang}} term for "Respondent".

resulted in “you,” while banning pronouns led to “Respondent.” To stabilize output, I used:

always use the fixed keyword "_RESPONDENT_"

and replaced it client-side 😅

const RESPONDENT = i18n.text({
    en : 'Respondent',
    ja : '回答者',
    fr : 'Répondant',
    de : 'Befragter',
    es : 'Encuestado',
    pt : 'Respondente',
    hi : 'उत्तरदाता',
    ko : '응답자',
    zh : '受访者'
});

Conclusion

Thank you for reading. In this project, I focused on structuring prompt tuning while controlling hallucination impact. Future improvements may include automated output quality checks:

• Rule-based validation of format and keywords
• Using one AI to evaluate another AI’s output

I hope my trial-and-error and trade-off decisions are helpful to you.