A beginner's guide to the Granite-8b-Code-Instruct-128k model by Ibm-Granite on Replicate

#coding #ai #machinelearning #programming

This is a simplified guide to an AI model called Granite-8b-Code-Instruct-128k maintained by Ibm-Granite. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.

Model overview

granite-8b-code-instruct-128k is a large language model developed by IBM Research. It is part of the Granite series of decoder-only code models that are trained on a vast corpus of code data across 116 programming languages. The Granite Code Instruct models are fine-tuned on a combination of permissively licensed instruction data to enhance their instruction following capabilities, including logical reasoning and problem-solving skills.

The model is similar to other Granite Code models like Granite-8B-Code-Instruct and Granite-34B-Code-Instruct, which also leverage the Granite-8B-Code-Base and Granite-34B-Code-Base models as starting points for fine-tuning on instruction data. These models demonstrate strong performance across diverse coding tasks, from code generation to explanation and translation.

Model inputs and outputs

Inputs

Prompt: The text prompt that the model should generate output for.
Top K: The number of highest probability tokens to consider for generating the output.
Top P: The probability threshold for generating the output.
Temperature: The value used to modulate the next token probabilities.
Presence penalty: A penalty applied to tokens based on their previous presence in the output.
Frequency penalty: A penalty applied to tokens based on their previous frequency in the output.
Seed: A random seed value to use for generating the output.
Max tokens: The maximum number of tokens the model should generate as output.
Min tokens: The minimum number of tokens the model should generate as output.
System prompt: A prompt to send to the model to help guide its behavior (for chat models).
Stop sequences: A comma-separated list of sequences to stop generation at.
Prompt template: A template to format the prompt with.