Sometimes a migration doesn't start with a plan — it starts with a broken build. That's exactly what happened to me: a .NET Framework 4.8 solution with 17 projects, grown over years, suddenly started failing intermittently in Azure DevOps. What came out of it was a full migration from packages.config/HintPath to PackageReference — plus a whole cluster of side quests that I want to write down here, because each one of them would have cost an afternoon of debugging on its own if you didn't already know the cause. The build is green again — whether everything also behaves correctly at runtime still has to be verified by testing.
Quick reference: symptom → cause
If you landed here with a specific error, here's the short version:
| Symptom | Cause | Section |
|---|---|---|
| "Package not found" on restore, though the package is in the feed |
packages.config deleted, references still in HintPath style |
The actual root cause |
| Restore fails only for internal packages | No <packageSources> in NuGet.config, or feed retention |
First lead / Side quest 2 |
| "Package not found" after the migration | Assembly name instead of package ID in Include
|
The trap |
NU1605 – package downgrade detected |
Normal consequence of strict dependency resolution | Expect NU1605 |
| Build green, but the wrong target framework is active | Targeting pack missing (runtime ≠ targeting pack) | Side quest 1 |
CA0063 / CA0064
|
Reference to a .ruleset that no longer exists |
CA0063/CA0064 |
| Shifting, non-reproducible restore errors | Race condition from a parallel restore job | Race condition |
CS8032 |
Analyzer requires newer Roslyn than the agent has | CS8032 |
MSB3277 |
Conflicting assembly versions | MSB3277 |
Starting point
The solution is a classic .NET Framework 4.8 codebase with 17 project files, referencing both third-party packages (EntityFramework 6.5.2, Azure.Core, Enterprise Library, xunit, various System.*) and internally maintained packages from a private NuGet feed. Build and deployment run through an Azure DevOps pipeline on self-hosted agents.
The trigger: NuGet restore started failing sporadically in the pipeline with "package not found" errors for EntityFramework.6.5.2 and others — but not consistently across all projects. Locally on the dev machine, everything built just fine.
First lead: missing restore instructions
The first suspicion — a missing NuGetToolInstaller or NuGetCommand task ahead of the actual build step — turned out to be part of the problem, but not the core of it. A look at the solution-local NuGet.config revealed something else: it only contained disableSourceControlIntegration, with no <packageSources> section at all. That worked locally because the internal feed was configured globally on the dev machine — the build agent had no such global configuration.
The fix: package sources belong explicitly in the solution-local NuGet.config, with a <clear /> in front:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
<add key="internal-feed" value="https://.../index.json" />
</packageSources>
<solution>
<add key="disableSourceControlIntegration" value="true" />
</solution>
</configuration>
The <clear /> is the crucial part. Without it, whatever sources are configured globally on the machine get merged in. That's precisely how you end up in the "green locally, red on the agent" situation — your own machine carries years of accumulated global entries that the build agent simply doesn't have. With <clear />, the source list is identical and reproducible for anyone who checks out the repository.
A quick way to see what's actually in effect on a given machine or agent:
nuget config -All
But the real root cause sat deeper still.
The actual root cause: deleted packages.config with HintPath references
During an earlier cleanup, the packages.config files of several projects had been deleted — while the .csproj files still referenced packages in the old HintPath style:
<Reference Include="EntityFramework, Version=6.0.0.0, ...">
<HintPath>..\packages\EntityFramework.6.5.2\lib\net45\EntityFramework.dll</HintPath>
</Reference>
With this reference style, NuGet absolutely requires packages.config as its restore instruction. Without it, NuGet has no information about what should be downloaded into the local packages folder. The HintPath keeps pointing at a folder that never gets populated — hence the seemingly random "package not found" errors, depending on what happened to already be sitting in that folder.
How to spot it: if a project contains <Reference> entries with <HintPath>..\packages\... but has no packages.config in the project folder (anymore), that's the unambiguous fingerprint of this problem. A quick check across the whole solution:
Get-ChildItem -Recurse -Filter "*.csproj" | ForEach-Object {
$dir = $_.DirectoryName
$hasHintPath = Select-String -Path $_.FullName -Pattern "<HintPath>" -Quiet
$hasPackagesConfig = Test-Path (Join-Path $dir "packages.config")
if ($hasHintPath -and -not $hasPackagesConfig) {
Write-Output "Affected: $($_.FullName)"
}
}
Everything this prints either needs its packages.config back, or needs to be migrated to PackageReference.
My call: rather than restoring packages.config, I went the cleaner, more modern route — migrating to PackageReference.
Why by hand and not with the migration assistant?
The obvious question first: there is a migration assistant — right-click packages.config or the References node, "Migrate packages.config to PackageReference". So why do all of this manually?
Two reasons, both of which applied here:
- The assistant is a Visual Studio feature (VS 2017 15.7 and later). I work in JetBrains Rider, and that path wasn't available to me in my setup.
-
The assistant operates on
packages.config— it reads that file as its input to derive the package list. And that file had been deleted in exactly the affected projects. So even with Visual Studio, the standard tool wouldn't have been applicable.
If you have an intact packages.config and you're working in Visual Studio: use the assistant, it saves you all the manual work below. The manual route in this article is plan B for when one of those two conditions isn't met — and that's precisely when the package ID trap in the next section is where things go wrong.
Migrating to PackageReference
PackageReference solves the structural problem for good: no more packages.config, no HintPath synchronization, packages managed centrally in the global NuGet cache (%userprofile%\.nuget\packages) and resolved through obj\project.assets.json. This works with classic .NET Framework projects too — not just SDK-style ones — as long as you're on NuGet 4.0+ / VS2017+.
Key settings I kept consistent across projects:
<PropertyGroup>
<RestoreProjectStyle>PackageReference</RestoreProjectStyle>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
</PropertyGroup>
- Framework assemblies (
System.*, etc.) stay as classic<Reference>entries — those aren't managed through NuGet. - The
EnsureNuGetPackageBuildImportstarget block and all the legacy.props/.targetsimports NuGet used to generate for thepackages.configstyle get removed. - Binding redirects are now generated automatically instead of hand-maintained in
App.config— which matters when duplicate versions are floating around (e.g.EntityFramework6.0.0 and 6.5.2 both sitting in the oldpackagesfolder).
Projects using EDMX/T4 code generation (EF6 Database First) needed extra attention, because the generated .Designer.cs files had to keep working — the plain PackageReference switch doesn't automatically cover that, so the codegen infrastructure needs to be verified alongside it.
I migrated all 17 project files one at a time rather than risking a big-bang migration. That approach transfers to any grown .NET Framework codebase: convert one project, commit and build it locally, then move to the next. With five, six or more affected projects, this drastically cuts down debugging time, because a build failure immediately tells you which project caused it.
Practical per-project workflow:
- Note every
<Reference>entry with a<HintPath>(the version is reliably in the path — the package name is not always, see the warning below). - Replace the
<Reference>block with one<PackageReference Include="PackageId" Version="x.y.z" />per package. - Leave
<Reference>entries for pure framework assemblies (no corresponding NuGet package, e.g.System.Data) untouched. - Remove the legacy NuGet-generated
.props/.targetsimports and theEnsureNuGetPackageBuildImportstarget block. - Set
RestoreProjectStyle,AutoGenerateBindingRedirectsandGenerateBindingRedirectsOutputType(see above). - Run
nuget restore/dotnet restorelocally and check thatobj\project.assets.jsonis generated cleanly — that's the most reliable signal the switch worked for this project. - Only then commit and move on to the next project.
The trap: assembly name is not package ID
This is the mistake that cost me the most time — and it's insidious, because it looks entirely plausible at first glance.
In the HintPath style, the Include attribute holds the assembly name. In PackageReference, it needs the NuGet package ID. For most packages these are identical (EntityFramework is called the same as an assembly and as a package) — but not for all of them. An example from my solution:
<!-- old: assembly name -->
<Reference Include="Microsoft.Practices.EnterpriseLibrary.Common">
<HintPath>..\packages\EnterpriseLibrary.Common.6.0.1304\lib\NET45\Microsoft.Practices.EnterpriseLibrary.Common.dll</HintPath>
</Reference>
<!-- new: package ID -->
<PackageReference Include="EnterpriseLibrary.Common" Version="6.0.1304" />
If you mechanically carry over the Include value, NuGet goes looking for a package called Microsoft.Practices.EnterpriseLibrary.Common — which doesn't exist in the feed. Result: another "package not found", this time self-inflicted. In my case this error propagated through 16 files and had to be undone with a batch replace.
The reliable source for the package ID is the folder name in the packages path, not the Include attribute: ..\packages\EnterpriseLibrary.Common.6.0.1304\... → package ID EnterpriseLibrary.Common, version 6.0.1304. That folder is always named <PackageId>.<Version>. So if you script the migration, parse the path — don't copy the Include attribute.
NU1605: why downgrade warnings show up after the migration
After the switch, several projects started producing NU1605 warnings ("detected package downgrade"). This isn't an edge case — it's the normal consequence of migrating. packages.config never strictly validated transitive dependencies; each project simply pulled whatever was in its own list. PackageReference resolves the full dependency graph strictly and flags every conflict where a direct reference forces an older version than a transitive dependency requires.
In practice that means: after migrating, version conflicts surface that were silently baked into the build before. The fix is usually unspectacular — raise all affected projects to the same higher, consistent version set. The important thing is not to be blindsided by it: these aren't new problems the migration created, they're old inconsistencies it made visible.
Once all projects were migrated and the versions aligned, dotnet restore completed with 0 errors and 0 warnings.
Cleanup: the packages folder is dead
One detail that tends to get left behind: after the migration, the solution-local packages folder is no longer populated and no longer read — all packages now live in the global cache under %userprofile%\.nuget\packages. So the old folder can be deleted, as soon as all projects have been converted. As long as one project is still on the HintPath style, that one still needs it.
If the folder was versioned until now, it also belongs in .gitignore:
# NuGet: no longer needed after the PackageReference migration
packages/
*.nupkg
And one safety net that sounds trivial but makes all the difference with 17 project files in play: the whole conversion ran on a feature branch, not on the main branch. That makes every intermediate state throwaway-able, which is what gives you the nerve to rewrite a whole file instead of cautiously patching around in it.
Side quest 1: the silently downgrading framework compiler
In parallel, the question came up whether bumping from v4.8 to v4.8.1 would be worthwhile. The runtime was there — confirmed via the registry:
(Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full").Release
# 533509 → corresponds to .NET Framework 4.8.1
The second check, however, was sobering:
Test-Path "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8.1"
# False
And this is the part that's easy to miss: the runtime and the targeting pack are two different things. The runtime only tells you that applications targeting 4.8.1 can run on this machine. To compile against 4.8.1, you additionally need the reference assemblies from the developer/targeting pack. So "but 4.8.1 is installed" — whether from Windows Features or the registry — doesn't answer the compile-time question at all.
That's why, even though the .csproj already declared v4.8.1 as its target, MSBuild silently fell back to v4.8 — no error, no warning. This is easy to miss precisely because the build stays green.
How to confirm it if you suspect it: search the MSBuild log (verbosity at least "Detailed") for the csc.exe invocation line — the reference paths will point at either ...\.NETFramework\v4.8\... or ...\.NETFramework\v4.8.1\.... Equally telling: the generated file in the obj folder named along the lines of .NETFramework,Version=v4.8.1.AssemblyAttributes.cs — the version part of that filename reflects the version actually used, regardless of what the .csproj claims.
Fix if the pack is missing:
- Install the .NET Framework 4.8.1 Developer Pack (requires admin rights).
- Verify afterwards:
Test-Path "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8.1"should now returnTrue. - Fully restart the IDE (not just reload the solution) — the reference assembly cache is read at IDE startup.
- Clean and full rebuild, not just incremental, so the generated
AssemblyAttributes.csfiles are recreated.
One important caveat: installing the developer/targeting pack locally has zero effect on your Azure DevOps build agents. That has to be handled separately — for Microsoft-hosted agents, check the image documentation for which frameworks come preinstalled; for self-hosted agents, install the developer pack there manually as well. Projects still targeting 4.8 keep working unaffected, since 4.8.1 is backwards compatible. In the end the conclusion was clear: the 4.8.1 bump is a nice-to-have with no urgency — the actual priority remained the restore fix.
Side quest 2: internal package versions and feed retention
On top of that, restore errors were showing up exclusively for internally maintained packages — not for EntityFramework or other third-party packages, which verified cleanly. Possible causes I worked through:
- The package version was never successfully pushed to the feed (failed publish pipeline).
- A typo or wrong version in the reference.
- Feed retention/cleanup: some internal feeds automatically prune old, unused package versions.
- Wrong feed scope (e.g. hitting the release view when the version only exists as prerelease).
Concrete checks before you go hunting through your own code:
- Look directly in the feed (the provider's web UI, or
nuget list <PackageName> -Source <FeedUrl> -AllVersions -PreRelease) to see whether the referenced version actually still exists. - Verify that the build/publish pipeline meant to produce that package version actually succeeded — not just that it was triggered.
- Compare the version in source (
.csproj, or previouslypackages.config) against what's available in the feed, exactly — including casing and suffixes like-beta. - If the feed has multiple views (e.g. "Release" vs. "Prerelease"/"Local"), make sure the build agent is hitting the view that actually contains the version you need.
A useful reminder: with internal feeds, do this comparison first, before spending days looking for the bug in your own code — the cause is almost never in the code itself.
Side quest 3: several pipeline problems in a single afternoon
Once the project structure was in place, the first full pipeline run surfaced several independent problems all at once.
CA0063/CA0064: stale ruleset reference
Several .csproj files still had <CodeAnalysisRuleSet> pointing at a .ruleset file that no longer existed, producing CA0063 ("ruleset file not found") and CA0064 warnings. You can spot this directly in the build output from those exact warning codes plus the path to the missing file. Fix: remove the <CodeAnalysisRuleSet> property from the affected .csproj files — either individually, or more cleanly by managing a shared analyzer/ruleset configuration centrally in a Directory.Build.props at the solution root instead of maintaining it per project.
YAML syntax error: the misindented task
A copy-paste mistake had accidentally indented a NuGetCommand@2 task underneath the script: scalar of a PowerShell task — a classic, hard-to-see YAML indentation bug. Azure DevOps usually reports these with a line number in the parse error, but the actual cause (wrong indentation depth) often sits a few lines above or below it. Worth doing: run the YAML through a linter/validator (Azure DevOps has a built-in YAML preview, or use an external YAML linter) before kicking off another build — it saves you waiting on the agent.
Race condition from a parallel restore job
A separate Restore job ran in parallel with the template-based build job, both hitting the same .nuget\packages folder — a textbook race condition. The build job template already brought its own restore step, so the separate job could simply be removed. A tell-tale sign of this kind of race condition in the logs: shifting, non-reproducible restore errors that come and go across re-runs of the same commit. That's unusual for a structural problem (like the deleted packages.config), so it points toward timing/concurrency instead. In general, when using pipeline templates, look inside them to see which restore/build steps they already provide before adding your own redundant jobs.
Side quest 4: CS8032 — when the analyzer is newer than the compiler
The most stubborn error came last: CS8032 when building an EF6/EDMX project that had no modern dependencies of its own. The error message names the required Microsoft.CodeAnalysis version — that's your most important clue. In my case, Azure.Core 1.60.0 transitively pulled in System.ClientModel 1.14.0, and that package ships Roslyn analyzers requiring Microsoft.CodeAnalysis 4.3.0.0. The build agent, however, was still on a VS 2019 toolset with Roslyn 3.11 — locally, with a newer toolset, the error never showed up. Same mechanism as the targeting pack: it isn't the code that differs, it's the environment.
How to find out which package brings the analyzer: search the global NuGet cache (%userprofile%\.nuget\packages) for analyzers\dotnet\cs\*.dll — the package folder above it identifies the culprit:
Get-ChildItem "$env:USERPROFILE\.nuget\packages" -Recurse -Directory -Filter "analyzers" |
Select-Object -ExpandProperty FullName
For SDK-style projects, dotnet list <project>.csproj package --include-transitive also works; for classic, non-SDK-style .csproj files — which is exactly the case here — it generally does not work reliably. That's why going through the package cache is the more dependable route in this scenario.
Option A — surgical, when only one or two projects are affected: reference the offending package explicitly and exclude only its analyzers:
<PackageReference Include="System.ClientModel" Version="1.14.0">
<ExcludeAssets>analyzers</ExcludeAssets>
</PackageReference>
This is the cleaner solution, because analyzers from every other package keep running. The downside: you have to locate and touch each affected project individually — and with a transitively pulled package, that's usually more projects than you'd expect. In my case, the error popped right back up in the next project after the first fix, then the one after that.
Option B — global, when the problem runs through the whole solution: since upgrading the agent toolset wasn't something I could do on short notice (the build server is owned by a different department), the pragmatic solution was a central Directory.Build.props at the solution root:
<Project>
<PropertyGroup>
<RunAnalyzersDuringBuild>false</RunAnalyzersDuringBuild>
</PropertyGroup>
</Project>
Important — and this is where I went down the wrong path at first: the obvious reflex is <NoWarn>$(NoWarn);CS8032</NoWarn>. That does nothing here. CS8032 shows up in the log as an error (##[error]CSC(0,0): Error CS8032), and NoWarn only suppresses diagnostics classified as warnings. For a hard compiler error — analyzer instantiation fails — it simply doesn't apply. The actual effect comes from RunAnalyzersDuringBuild=false, because that stops csc from loading the analyzer DLLs in the first place.
Where Directory.Build.props has to live: MSBuild searches upward from each project folder and stops at the first match. So it has to sit somewhere that is a parent directory of all affected project folders — usually next to the .sln. A common stumbling block: some IDEs don't show the file in the Solution Explorer automatically. That's normal and doesn't mean it isn't taking effect — it doesn't need to be included as a solution item.
The big advantage of the global approach: you don't have to hunt down and patch every single project that transitively pulls in the problematic package — including the ones you haven't even noticed yet. After adding it, still check the log to confirm CS8032 is gone for all projects, not just the one you started with.
MSB3277: conflicting assembly versions
At the same time, an MSB3277 warning appeared about conflicting System.Text.Json versions. Same root cause — overly modern packages pulled in transitively into a net48 project — and it can be defused via the automatically generated binding redirects mentioned earlier. If you're seeing CS8032, search the same log for MSB3277 too.
What a green build does not prove
An honest status update at this point: the build passes, and dotnet restore reports 0 errors and 0 warnings. What that does not prove is that everything works correctly at runtime. The compiler checks references and syntax — it doesn't start the application. Verifying the behaviour is still ahead of me.
This migration in particular shifts several things that only surface at runtime:
-
Binding redirects are now generated automatically instead of being hand-maintained in
App.config. If the generated version differs from the old one, you won't find out at compile time — you'll find out as aFileLoadExceptionorFileNotFoundExceptionthe first time the affected assembly is loaded. - Raised package versions (the NU1605 resolution) mean newer code is now running in several places. It compiles — whether behaviour changed is not something the build can tell you.
-
EF6 loads its provider at runtime via
App.config. If an entry is missing after the switch, or points at a different version, you'll discover it on the first database access. -
Content files from packages were sometimes copied to the output directory differently under
packages.configthan underPackageReference. Again, purely a runtime concern.
What's on the verification list:
- Actually start the application, not just build it.
- Trigger a real database access through the EF context.
- Diff the generated
.configin the output directory (bin\...\*.exe.config) against the old, hand-maintained version — that shows at a glance which binding redirects were added, dropped, or repointed. - Run the test suite, paying particular attention to the projects whose package versions were raised.
This sounds obvious, but in practice it isn't: after a week of a red pipeline, the temptation to file the green checkmark as "done" is considerable. It's really just the point where the actual testing can begin.
Takeaways
What started as a single "package not found" error turned out to be a chain of independent but mutually reinforcing problems: a structural error (deleted packages.config with HintPath references), an environment gap between the local machine and the build agent (targeting pack, toolset version, NuGet sources), a race condition in the pipeline design, and several smaller configuration mistakes — plus one self-inflicted wound from confusing assembly names with package IDs. The build is green now; the runtime verification, as described above, is still pending.
The main lesson: green locally does not mean CI-ready. With grown .NET Framework solutions in particular, it pays to explicitly ask what only works on your own machine "by accident", because something is configured globally somewhere that the build agent doesn't have. Three of the problems described here — missing package sources, missing targeting pack, outdated Roslyn toolset — are the exact same mechanism wearing three different costumes. Once you've recognized that pattern, the next "but it works on my machine" doesn't send you into the code first.
In the end, migrating to PackageReference wasn't just the fix for the immediate problem — it made the entire restore chain substantially more robust: no more path dependencies, no more double bookkeeping between .csproj and packages.config, and transitive dependencies resolved automatically instead of hundreds of individually maintained references. The NU1605 warnings that surfaced along the way weren't a cost of the migration — they were the migration doing its job.
What's next
Central Package Management. With 17 projects on PackageReference, every package version now lives in 17 files — exactly the problem I just solved by hand while aligning the NU1605 conflicts. With a Directory.Packages.props and ManagePackageVersionsCentrally, each version is declared once centrally, and the .csproj only carries <PackageReference Include="..." /> without a version. The next version bump becomes one line instead of seventeen.
Lockfiles. RestorePackagesWithLockFile writes the resolved dependency graph into a packages.lock.json that gets committed alongside the code. That makes restore on the build agent bit-for-bit identical to the local one — and any divergence surfaces as an error instead of disguising itself as "but it works on my machine". For an article whose whole thread is that exact difference, it's the logical endpoint.
On how this came about: I worked through the debugging and the migration with the support of an AI assistant, and used it again to assemble this article from my working notes. Every error, fix and dead end described here comes from the actual course of the project — including the places where we took a wrong turn together.
Top comments (0)