Packets and Frames in libav transcoding pipeline

I found it a bit confusing to identify when to use AVFrame and when to use AVPacket while working with libav.

TLDR; Packets contain encoded data. Frames contain decoded data

That is the only difference between packets and frames but it is the core reason for their differences in attributes and the stage in which they are produced in a transcoding pipeline.

Let us understand this with an example

Taking referrence from the famous libav tutorial on github

The AVFormatContext is the abstraction for the format of the media file, aka container (ex: MKV, MP4, Webm, TS). The AVStream represents each type of data for a given format (ex: audio, video, subtitle, metadata). The AVPacket is a slice of compressed data obtained from the AVStream that can be decoded by an AVCodec (ex: av1, h264, vp9, hevc) generating a raw data called AVFrame.

To implement this, we use the following functions (in this order)

• avcodec_send_packet to send an encoded packet to the decoder
• avcodec_receive_frame to recieve a raw frame from the decoder

We then perform any modifications such as scaling, resampling, cropping, format conversion, etc. on the raw frame.
Remember to keep the frames compatible for the encoder being used.

• avcodec_send_frame to send a raw frame to the encoder
• avcodec_receive_packet to recieve an encoded packet from the encoder

But this also has further intricacies like:

Multiple frames in single packet

A (encoded) packet could contain multiple (raw) frames while. This can occur separately while encoding and decoding and depends on the attributes of the codec (AVCodecContext).
Hence it is a good idea to use avcodec_receive_frame inside a loop and read frames till it does not return an error, specifically AVERROR(EAGAIN), that signals that the internal buffer (of the codec) is now empty.

avcodec_decode_* and avcodec_encode_* methods

It can be confusing to find these methods being used in other libav projects/scripts to build a complete transcoding workflow with fewer function calls.
But, as mentioned here the above mentioned workflow of packets and frames is meant to replace these legacy functions.