A beginner's guide to the Granite-20b-Code-Instruct-8k model by Ibm-Granite on Replicate

#coding #ai #machinelearning #programming

This is a simplified guide to an AI model called Granite-20b-Code-Instruct-8k maintained by Ibm-Granite. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.

Model overview

The granite-20b-code-instruct-8k model is part of the Granite series of decoder-only code models developed by IBM Research. These models are trained on a vast dataset of over 3 trillion tokens of code across 116 programming languages, as well as additional high-quality natural language datasets. The key advantage of the Granite Code models is their ability to excel at a wide range of code-related tasks, including code generation, explanation, fixing, editing, and translation. The granite-20b-code-instruct-8k model is further fine-tuned on instruction-following datasets to enhance its capabilities in areas like logical reasoning and problem-solving.

The Granite Code models come in a range of sizes, from 3B to 34B parameters, allowing users to choose the variant that best fits their needs and resources. Similar models in the Granite family include the granite-8b-code-instruct, granite-34b-code-instruct-8k, and granite-3.0-8b-instruct.

Model inputs and outputs

Inputs

Prompt: The text that the model should use as a starting point for its generation.
Min Tokens: The minimum number of tokens the model should generate as output.
Max Tokens: The maximum number of tokens the model should generate as output.
Temperature: The value used to modulate the next token probabilities, controlling the level of randomness in the generated output.
Top K: The number of highest probability tokens to consider for generating the output.
Top P: The probability threshold for generating the output, where only the top tokens with cumulative probability greater than or equal to this value are considered.
Frequency Penalty: A penalty applied to the model's likelihood of repeating the same tokens.
Presence Penalty: A penalty applied to the model's likelihood of generating new tokens.
Stop Sequences: A comma-separated list of sequences that will stop the generation if encountered.