How we got here
Before the end of 2021, my work scheduled a hackathon for my organization. With the requirement being that it be useful for us or our players. I decided to attempt to develop a language server to help engineers get their services into our production stack.
Our stack is pretty complex and takes a lot of learning. Although the language one interacts with is yaml, it's really a DSL and requires a lot of support. The goal of this tool was to help cut down the amount of assistance someone would need who is new / generally unfamiliar with the stack.
I found it pretty difficult to get up and running, so I wanted to share my work and progress for any other developer productivity enthusiasts!
What is a Language Server
A language server is a standard developed by Microsoft that defines a language server and language client; which handle the language syntax parsing, linting, code hints (etc) and the actual interfacing with the developer respectively. Two language server clients are Visual Studio Code, and recently Neovim on version 0.6 (written entirely in Lua!). Some language servers include yamlls, sourcegraph-go and rust-analyzer. The language server protocol then defines the glue between these two components. The glue happens over JSON-rcp.
For our custom LSP, we'll start by forking the yaml-language-server maintained by redhat.
Getting started
The first thing you'll need to do is enable the LSP for your editor of choice. I'm very deep into the vim camp, so I did neovim. This is the typical setup code needed to run the yamlls
local nvim_lsp = require('lspconfig')
local servers = {
'yamlls',
}
for _, lsp in ipairs(servers) do
nvim_lsp[lsp].setup {
on_attach = on_attach,
flags = {
debounce_text_changes = 150,
}
}
end
Which just tells the neovim LSP to setup the client to talk to the language server. The yaml-language-server runs a binary on your local computer. The default is with the --stdio flag, which would accept standard input and validate: yaml-language-server --stdio. If you run :LspInfo from vim, you'll see this info
1 client(s) attached to this buffer: yamlls
Client: yamlls (id 1)
root: /Users/ischweer/dev/shards
filetypes: yaml
cmd: yaml-language-server --stdio
There tends to be a lot of config you can set as well, you could also run the language server separately with an --rpc-port and --rpc-host option if you chose to run the language server in one window, and test it in the other.
A small demo of what I built
You're ready to start developing. In my case, I forked the yaml-language-server, and started adding some addition logic to fetch application specs, their configuration, and WAN setup from our central services. A service at Riot has this sort of kube-esque definition, just with more business DSL ontop
application-instance:
name: some.app
network:
inbound:
- name: another.app
location: usw2
Pulling all these down over http is fairly straight forward. Once we have all the yamls, they are cached
async downloadApps() {
// we want to go through all the apps in the env, and get
// the yaml'd app definitions
await this.getDiscoverous();
// now we can get the lol-services 710e env
const url = `https://${this.gandalf_host}/api/v1/environments/lol-services/${env.LATEST_VERSION}`;
const resp: XHRResponse = await xhr({ url: url, headers: { authorization: `Bearer ${this.gandalf_token}` } });
console.log('Successfully grab 710e instance');
const cache_builder: Array<Promise<boolean>> = [];
for (const appMetadata of environmentInstance['environment']['applications']) {
cache_builder.push(this.downloadApp(appMetadata));
}
await Promise.all(cache_builder);
}
async downloadApp(app: { name: string; version: string }): Promise<any> {
const url = `https://${this.gandalf_host}/api/v1/applications/${app.name}/${app.version}`;
const resp = await xhr({ url: url, headers: { authorization: `Bearer ${this.gandalf_token}` } });
const _d = new YAML.Document();
_d.contents = JSON.parse(resp.responseText);
return new Promise(() => {
fsp
.writeFile(`${homedir()}/.cache/gandalf/${app.name}.yaml`, _d.toString(), { flag: 'wx' })
.then()
.catch((err) => {
console.log(`Did not write file because it exists already ${app.name}.yaml`);
});
});
}
In order to fully understand this yaml parsing, you'll need to read the subsequent blog post on the riot eng blog :) however, once it's downloaded, we have now have some cached files that help us answer questions like "Is service X actually specified talk to service Y, or is it a typo?". That looks like this in the typescript
for (const defined_outbound of appSpec.outbounds || []) {
let found = false;
for (const _instanced_outbound of app_instance_outbounds.items || []) {
const instanced_outbound = _instanced_outbound as YAMLMap;
if (instanced_outbound.get('service') == defined_outbound) {
found = true;
break;
}
}
if (!found) {
errors.push({
message: `Missing required outbound for ${defined_outbound}`,
location: { start: app_instance_outbounds.range[0], end: app_instance_outbounds.range[2], toLineEnd: true },
severity: 1,
source: 'Gandalf',
code: ErrorCode.Undefined,
} as YAMLDocDiagnostic);
}
}
With that in place, we can see nice little errors in our editor when services are expected to talk to each other, but not specified to do so.
Actually doing the work
LSP is all over RPC, and you can see all the events specified above. Everything is async, and the client needs to be able to handle json serialized text for everything. You need to learn your editors internals to fully comprehend how the LSP is working :').
For neovim, always check the lsp logs: tail -n 10000 ~/.cache/nvim/lsp.log | bat. They will be extremely valueable, as most times you'll have serialization errors or unknown property issues. I found adding the following "always log" approach helped when debugging (though it slows the editor down)
console.error = (arg) => {
if (arg === null) {
connection.console.info(arg);
} else {
connection.console.error(arg);
}
};
this will let you write more normal typescript code, read the resulting errors in the log, and then have a bit of a tighter feedback loop.
Top comments (0)