DEV Community: Real Actioner

When 100.00 Means Nothing: Gaming Coding Assessments

Real Actioner — Mon, 23 Mar 2026 18:55:40 +0000

I recently worked on a machine learning challenge on HackerRank and got a strong score with a real model. Then I noticed something frustrating: some top-scoring submissions appeared to hardcode outputs for known hidden tests instead of solving the problem algorithmically.

This is not just a leaderboard issue. It is an assessment integrity issue.

Problem link: Dota 2 Game Prediction (HackerRank)

The Problem in One Line

If a platform can be gamed by memorizing test cases, the score stops measuring skill.

A Visual Difference in Code

Here is what a genuine solution path looks like (train on trainingdata.txt, build features, fit a model, then predict):

train_df = pd.read_csv(TRAINING_FILE, names=list(range(11)))
hero_categories = list(set(train_df.iloc[:, : 2 * TEAM_SIZE].values.flatten()))

train_t1, train_t2 = build_team_features(train_df, hero_categories)
train_matrix = pd.concat([train_t1, train_t2, train_df.iloc[:, -1]], axis=1)

model = RandomForestClassifier(n_estimators=MODEL_TREES, random_state=MODEL_RANDOM_STATE)
model.fit(train_matrix.iloc[:, :-1], train_matrix.iloc[:, -1])

test_df = read_test_rows()
test_t1, test_t2 = build_team_features(test_df, hero_categories)
test_matrix = pd.concat([test_t1, test_t2], axis=1)
predictions = model.predict(test_matrix)

And here is the anti-pattern (hardcoded expected outputs by test size, no real inference):

K = int(input())

res_100 = [2, 1, 1, ...]
res_3000 = [2, 1, 2, ...]

if K == 100:
    for i in res_100:
        print(i)
elif K == 3000:
    for i in res_3000:
        print(i)

The second snippet can score high on fixed tests, but it does not solve the problem in a reusable or trustworthy way.

Why This Matters

1) Platforms Must Detect This Behavior

Assessment platforms have a responsibility to ensure their tests measure problem-solving ability, not test-set leakage or lookup-table tricks.

When a fixed hidden dataset is reused too long, it becomes vulnerable. Once leaked, candidates can optimize for those exact cases and still appear "excellent" on paper.

2) Developers Should Be Honest About Skill

A high score obtained through memorization is not equivalent to engineering competence.

Short-term leaderboard wins can become long-term career risk:

You may pass a filter you are not ready for.
You may underperform in real tasks where no leaked answers exist.
You may damage trust with teams and employers.

Ethics in engineering is not only about production systems. It starts with how we represent our own abilities.

3) Honest Developers Get Penalized Otherwise

When dishonest strategies are rewarded, honest candidates are pushed down rankings despite better fundamentals.

That creates a harmful signal:

"Gaming beats learning."
"Memorization beats reasoning."
"Optics beat capability."

Over time, this hurts both developers and hiring quality.

What Platforms Can Do (Practical Fixes)

Assessment quality can improve dramatically with better test design and anti-abuse checks:

Frequent hidden test rotation

Avoid static hidden sets that remain unchanged for long periods.
Randomized or generated test cases

Use input generation with controlled distributions to reduce memorization value.
Perturbation checks

Run near-duplicate and slightly modified versions of hidden cases. Hardcoded solutions often fail immediately.
Generalization scoring

Reward robustness across multiple unseen shards, not a single hidden file.
Suspicion heuristics

Flag submissions with patterns like exact-case branching, massive literal maps, or unusual I/O fingerprints.
Code review signals

Include basic static checks for algorithm presence and complexity plausibility.

What Hiring Teams Can Do

Do not rely on a single challenge score.

Use layered evaluation:

coding exercise score
solution walkthrough and trade-off discussion
debugging or extension task on the same code
communication and reasoning quality

A candidate who truly understands their solution can adapt it under new constraints.

What Developers Should Do

Build real solutions, even if your score is not perfect.
Optimize for transferable skill, not exploitability.
Be transparent about what you know and where you are still learning.

An honest 92 with a genuine approach is often more valuable than a gamed 100.

Final Thought

Assessment platforms and developers share responsibility.

Platforms should design systems that reward real problem solving.

Developers should choose integrity over shortcuts.

If we fail on either side, honest engineers lose, and hiring signals become noisy.

If we improve both sides, scores can become meaningful again.

This is also why I decided to build my own assessment platform: one that is explicitly designed to reward generalization, reasoning, and engineering integrity instead of fixed-test memorization.

From 6,000 Competitors to Top 3 in 30 Minutes

Real Actioner — Mon, 23 Mar 2026 15:52:01 +0000

I finished 3rd out of about 6,000 competitors on a HackerRank challenge in the AI -> Statistics and Machine Learning track, and the final version of my solution came together in roughly 30 minutes.

If you enjoy problem solving in this space, welcome - this write-up is for you.

Problem link: Matching Questions with their Answers

If you are interested in statistics/machine-learning style reasoning problems, this one is a great example: it looks like simple text matching, but it really tests scoring, disambiguation, and robust decision-making under noisy input.

What looked easy on paper turned out to be surprisingly tricky in practice: five questions, five shuffled answers, and one paragraph where multiple answers can seem valid. The difference-maker was not a giant algorithm - it was combining a few solid ideas quickly and cleanly.

The Problem

Input has:

one paragraph,
five factual questions,
five candidate answers in random order.

Output must be the correct answer for each question, in question order.

All valid outputs must be exact substrings from the provided paragraph, which is where the matching strategy becomes interesting.

What I Did Differently

1) I used context scoring, not plain keyword overlap

A naive solution counts common words and guesses. That breaks when two options share similar wording.

So I scored each question-answer pair using:

keyword overlap in the same sentence,
keyword overlap in a local window around the answer,
distance from question keywords to the answer location in the paragraph.

This was the first big jump in accuracy.

def question_keywords(question):
    return [t for t in tokenize(question) if t not in STOPWORDS]

def build_keyword_positions(paragraph_lower):
    positions = {}
    for m in re.finditer(r"[a-z0-9']+", paragraph_lower):
        tok = m.group(0)
        positions.setdefault(tok, []).append(m.start())
    return positions

2) I added simple intent rules for common question types

I noticed recurring patterns and gave them light heuristic boosts:

how far, how long, height -> prefer numeric answers,
highest point -> prefer a named entity over a measurement,
lowest point -> prefer a measurement,
called / known as -> prefer label-like text instead of numeric values.

Small rules, big effect on tie-breaks.

if "height" in ql or "how far" in ql or "how long" in ql:
    score += 3.0 if has_digit else -2.5
if ("called" in ql or "known as" in ql) and has_digit:
    score -= 2.0
if "which is the highest point" in ql or "highest point in" in ql:
    score += -2.2 if has_digit else 1.5
if "lowest point" in ql and has_digit:
    score += 2.0

3) I anchored "What is X?" questions to the actual entity

Questions like What is INSEAD? and What is ESIGETEL? can be confused if both answers are descriptive.

So I explicitly extracted the target entity and boosted candidates near that mention:

extracts the target entity (INSEAD, ESIGETEL),
boosts answers near the entity mention in the paragraph,
adds extra score if both appear in the same sentence.

That cut down wrong swaps between definition-style answers.

def extract_target_entity(question):
    m = re.search(r"\b(?:what|who|where|when)\s+is\s+(.+?)\s*\?\s*$", question, flags=re.I)
    if not m:
        return ""
    return m.group(1).strip(" .")

# inside scoring
if ent_l and ent_pos != -1:
    ent_center = ent_pos + len(ent_l) // 2
    dist = abs(ent_center - center)
    score += 4.0 / (1.0 + dist / 80.0)

4) I used exact assignment instead of greedy matching

Greedy matching can lock into a bad early pick. Since there are only five answers, I just checked all permutations (5! = 120) and selected the max total score.

For this problem size, brute force is clean, exact, and very fast.

best_perm = None
best_score = -10**18
for perm in itertools.permutations(range(n)):
    total = 0.0
    for i, j in enumerate(perm):
        total += score_matrix[i][j]
    if total > best_score:
        best_score = total
        best_perm = perm

5) I hardened the solution for real test noise

Two practical fixes helped with stubborn edge cases:

normalize common mojibake patterns before matching,
score across all occurrences of an answer in the paragraph (not just the first one).

These details mattered more than expected on hidden tests.

def normalize_text(text):
    fixed = text
    if any(marker in fixed for marker in ("\u00C2", "\u00C3", "\u00EF\u00BF\u00BD", "\uFFFD")):
        for enc in ("latin-1", "cp1252"):
            try:
                fixed = fixed.encode(enc).decode("utf-8")
                break
            except Exception:
                pass
    return fixed.replace("\ufffd", "")

Why This Ranked High enough

This rank was not about one clever hack. It came from combining:

semantic/context scoring,
targeted heuristics,
exact combinational optimization,
defensive text normalization.

That gave me both visible-test accuracy and hidden-test stability, which is exactly what leaderboard positions reward.

Final Thoughts

Top-3 out of 6,000 in about 30 minutes was a strong reminder: in competitive coding, practical engineering choices often matter as much as core algorithm logic.

I did not overcomplicate the approach. I kept it understandable, added targeted scoring logic, and made sure edge cases would not break it. That balance is what carried this solution to the top.

If I get more time, I will keep competing in this track and share more breakdowns like this one, so feel free to follow for upcoming problem-solving articles.