DEV Community: 张智豪

Displaying images on a TFT screen using a ROM IP core

张智豪 — Thu, 06 Feb 2025 08:33:21 +0000

In a previous blog post, we learned how to display color bars on a TFT screen. In this post, we will take it a step further and learn how to display any image on the TFT screen. We will learn how to use a ROM IP core, write the corresponding module, and finally display the target image.

Click here to get the complete files of the code provided in this post.

Calling the ROM IP Core

To display an image, we need to convert the two-dimensional image into one-dimensional data and store it in ROM in a specific format. During operation, the TFT display reads data from the ROM core.

Image Format Conversion

Here we use an image conversion tool. Click here to get the software directly. The extraction code is 9jk4. Once you have the software, we can begin.

Find the image you want to display and convert it to a BMP bitmap.

The image I will display here is as follows:

You may not know how to convert JPG, PNG, or other image formats to BMP format. Here I will share my method, hoping it will be helpful to you.

Open the system's built-in Paint software and import the target image.

Process the image to ensure the size is appropriate.

Export the image.

Now that the image format issue is resolved, let's move on to the second step.

Get the .coe file

Open the software tool, load the image file, select the specific format (RGB565), and convert it.

I want to make a few points here:

RGB565: This is because our TFT display uses the RGB565 format.
Coe: The ROM IP core reads data, which is generally loaded with a .coe file.

Loading complete

Note down the size of the image: 169*267, as this will be needed later when configuring the ROM. Also, consider copying and pasting the generated file into the Vivado file path.

Configuring the ROM IP Core

Now let's configure the ROM IP.

Click the Window tab, find IP Catalog, search for ROM, find it, and double-click it.

Select Single Port ROM.

Configure the Option page.

Width: Represents the number of bits of stored data. For RGB565, it is 16 bits.
Depth: Represents the number of stored data, corresponding to the size of the image. In this case, 169 * 267 = 45123.

We choose Always Enable so that we can always read data from the ROM. Note: In actual project engineering, it is best to use an enable signal for control. Here, for the sake of simplicity, we choose to always enable it.

Do not check Primitives Output Register: If checked, the data will be two clock cycles slower than the enable signal. If not checked, the data lags by one clock cycle.

Load the data file.

If the Depth configured earlier is less than the number of data in the file, an error will be reported, as shown below:

After ensuring everything is correct, click the OK button, and then click the Generate button.

Image_Extract Module

▌Module Definition and Parameters Section

module Image_Extract
# (
  parameter H_Visible_area = 800, // Screen visible area width (pixels)
  parameter V_Visible_area = 480, // Screen visible area height (pixels)
  parameter IMG_WIDTH = 169,      // Original image width
  parameter IMG_HEIGHT = 267,     // Original image height
  parameter IMG_DATA_WIDTH = 16,  // Pixel data width (RGB565 format)
  parameter ROM_ADDR_WIDTH = 16   // Image ROM address bus width
)

The first two parameters describe the physical resolution of the screen.
The middle three parameters describe the characteristics of the stored original image.
The last two parameters define the bit width of the data bus and storage address.

▌Input and Output Port Definitions

(
  input clk_ctrl,                // TFT control clock (synchronized with the screen refresh rate)
  input reset_n,                 // Low-level reset signal
  input [15:0] img_disp_hbegin,  // Image display starting X coordinate
  input [15:0] img_disp_vbegin,  // Image display starting Y coordinate
  input [IMG_DATA_WIDTH - 1:0] disp_back_color,  // Background color (RGB value)
  input frame_begin,             // Frame synchronization signal (pulse at the beginning of each frame)
  input disp_data_req,           // Data request signal (valid display area flag)
  input [11:0] visible_hcount,   // Current horizontal scan position
  input [11:0] visible_vcount,   // Current vertical scan position
  input [IMG_DATA_WIDTH - 1:0] rom_data,  // Pixel data read from ROM
  output reg [ROM_ADDR_WIDTH - 1:0] rom_addra,  // ROM read address
  output [IMG_DATA_WIDTH - 1:0] disp_data     // Final output pixel data
);

Input signals are strictly synchronized with the TFT controller timing.
Use visible_hcount / visible_vcount for screen coordinate positioning.
disp_back_color is used for background filling in non-image areas.

▌ Boundary Detection Logic

// Detect if the image exceeds the screen boundary
assign h_exceed = img_disp_hbegin + IMG_WIDTH > H_Visible_area - 1;
assign v_exceed = img_disp_vbegin + IMG_HEIGHT > V_Visible_area - 1;

Calculate the theoretical right boundary of the image (X start + image width).
Compare whether it exceeds the maximum screen coordinates (H_Visible_area - 1).
Handle the vertical boundary in the same way.

▌ Display Area Determination Logic

// Generate a valid display area signal
assign img_h_disp = h_exceed ?
                    (visible_hcount >= img_disp_hbegin && visible_hcount < H_Visible_area) :
                    (visible_hcount >= img_disp_hbegin && visible_hcount < img_disp_hbegin + IMG_WIDTH);

assign img_v_disp = v_exceed ?
                    (visible_vcount >= img_disp_vbegin && visible_vcount < V_Visible_area) :
                    (visible_vcount >= img_disp_vbegin && visible_vcount < img_disp_vbegin + IMG_HEIGHT);

assign img_disp = disp_data_req && img_h_disp && img_v_disp;

When the image exceeds the screen, automatically limit the valid area to within the screen.

Normal mode(h_exceed= 0)

Effective condition: The picture is completely within the horizontal range of the screen.
Valid area: [img_disp_hbegin, img_disp_hbegin + IMG_WIDTH)
Example:
- Screen width 800
- Set starting coordinate = 100, image width = 200
- Valid X range: 100 ≤ X < 300

Truncation mode( h_exceed= 1)

Effective condition: The right side of the image exceeds the screen.
Valid area: [img_disp_hbegin, H_Visible_area)
Example:
- Screen width 800
- Set starting coordinate = 700, image width = 200
- Valid X range: 700 ≤ X < 800 (actually only displays the 100 pixels on the right)
  - The combined img_disp signal indicates that the current scan position needs to display image content.

▌ ROM Address Generation Logic

always @(posedge clk_ctrl or negedge reset_n) begin
  if (!reset_n)
    rom_addra <= 'd0;
  else if (frame_begin)
    rom_addra <= 'd0;
  else if (img_disp) begin
    if (visible_hcount == hcount_max)
      rom_addra <= rom_addra + (img_disp_hbegin + IMG_WIDTH - hcount_max);
    else
      rom_addra <= rom_addra + 1'b1;
  end
end

Reset the address to zero at reset or new frame start.
Increment the address pixel by pixel in the valid image area.
Special boundary handling: When scanning to the end of the line, calculate the address offset.

Normal situation: hcount_max = img_disp_hbegin + IMG_WIDTH - 1
Out-of-bounds situation: hcount_max = H_Visible_area - 1
Offset = Original planned next line starting address - Actually reachable address

▌ Data Output Logic

assign disp_data = img_disp ? rom_data : disp_back_color;

Output selection strategy:

Output the pixel data read from ROM in the image display area.
Display the background color in non-image areas.
Achieve real-time mixing of the image layer and the background layer.

▌Complete Code

module Image_Extract
# (
  parameter H_Visible_area = 800,   // Width of the entire screen display area
  parameter V_Visible_area = 480,   // Height of the entire screen display area
  parameter IMG_WIDTH = 169,        // Image width
  parameter IMG_HEIGHT = 267,       // Image height
  parameter IMG_DATA_WIDTH = 16,    // Image pixel width
  parameter ROM_ADDR_WIDTH = 16     // Storage image ROM address width
)
(
  input clk_ctrl,                          // Input clock, consistent with the TFT screen clock
  input reset_n,                           // Reset signal, active low

  input [15:0] img_disp_hbegin,            // Row coordinate of the first pixel in the upper left corner of the image to be displayed on the TFT screen
  input [15:0] img_disp_vbegin,            // Field coordinate of the first pixel in the upper left corner of the image to be displayed on the TFT screen
  input [IMG_DATA_WIDTH - 1:0] disp_back_color,  // Background color
  input frame_begin,                       // Flag signal for the start of a frame image, clk_ctrl clock domain
  input disp_data_req,                     // Data valid area
  input [11:0] visible_hcount,             // TFT visible area row scan counter
  input [11:0] visible_vcount,             // TFT visible area field scan counter
  input [IMG_DATA_WIDTH - 1:0] rom_data,    // Read image data
  output reg [ROM_ADDR_WIDTH - 1:0] rom_addra,  // ROM address for reading image data
  output [IMG_DATA_WIDTH - 1:0] disp_data      // Data displayed on the TFT screen
);

  wire h_exceed;
  wire v_exceed;
  wire img_h_disp;
  wire img_v_disp;
  wire img_disp;
  wire [15:0] hcount_max;

  // Determine if the image will exceed the screen range, resulting in incomplete display
  assign h_exceed = img_disp_hbegin + IMG_WIDTH > H_Visible_area - 1'b1;
  assign v_exceed = img_disp_vbegin + IMG_HEIGHT > V_Visible_area - 1'b1;

  assign img_h_disp = h_exceed ? (visible_hcount >= img_disp_hbegin && visible_hcount < H_Visible_area) :
                               (visible_hcount >= img_disp_hbegin && visible_hcount < img_disp_hbegin + IMG_WIDTH);

  assign img_v_disp = v_exceed ? (visible_vcount >= img_disp_vbegin && visible_vcount < V_Visible_area) :
                               (visible_vcount >= img_disp_vbegin && visible_vcount < img_disp_vbegin + IMG_HEIGHT);

  assign img_disp = disp_data_req && img_h_disp && img_v_disp;

  assign hcount_max = h_exceed ? (H_Visible_area - 1'b1) : (img_disp_hbegin + IMG_WIDTH - 1'b1);

  always @(posedge clk_ctrl or negedge reset_n) begin
    if (!reset_n)
      rom_addra <= 'd0;
    else if (frame_begin)
      rom_addra <= 'd0;
    else if (img_disp) begin
      if (visible_hcount == hcount_max)
        rom_addra <= rom_addra + (img_disp_hbegin + IMG_WIDTH - hcount_max);
      else
        rom_addra <= rom_addra + 1'b1;
    end else
      rom_addra <= rom_addra;
  end

  assign disp_data = img_disp ? rom_data : disp_back_color;

endmodule

Top Level Module

Header File

disp_param.vh

`define HW_TFT50

// Use VGA monitor, default is 640*480 resolution, 24-bit mode, other resolutions or 16-bit mode can be reconfigured in code lines 63 to 75
//`define HW_VGA

//=====================================
// The following macro definitions are used to set the bit mode and resolution based on the display device
//=====================================
`ifdef HW_TFT43  // Use 4.3 inch 480*272 resolution display
  `define MODE_RGB565
  `define Resolution_480x272 1 // Clock is 9MHz

`elsif HW_TFT50  // Use 5 inch 800*480 resolution display
  `define MODE_RGB565
  `define Resolution_800x480 1 // Clock is 33MHz

`elsif HW_VGA    // Use VGA monitor, default is 640*480 resolution, 24-bit mode
//=====================================
// You can choose other resolutions and 16-bit mode, users need to set according to actual needs
// The three lines and four lines below set the bit mode
// The continuous macro definition part after the five lines below sets the resolution
//=====================================
  `define MODE_RGB565
 // `define MODE_RGB888
  `define Resolution_640x480   1 // Clock is 25.175MHz
  //`define Resolution_800x600   1 // Clock is 40MHz
  //`define Resolution_1024x600  1 // Clock is 51MHz
  //`define Resolution_1024x768  1 // Clock is 65MHz
  //`define Resolution_1280x720  1 // Clock is 74.25MHz
  //`define Resolution_1920x1080 1 // Clock is 148.5MHz
`endif

//=====================================
// For non-special needs, users do not need to modify the following content
//=====================================
// Define different color depths
`ifdef MODE_RGB888
  `define Red_Bits   8
  `define Green_Bits 8
  `define Blue_Bits  8

`elsif MODE_RGB565
  `define Red_Bits   5
  `define Green_Bits 6
  `define Blue_Bits  5
`endif

// Define timing parameters for different resolutions
`ifdef Resolution_480x272
  `define H_Total_Time    12'd525
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd2
  `define H_Sync_Time     12'd41
  `define H_Back_Porch    12'd2
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd286
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd2
  `define V_Sync_Time     12'd10
  `define V_Back_Porch    12'd2
  `define V_Top_Border    12'd0

`elsif Resolution_640x480
  `define H_Total_Time    12'd800
  `define H_Right_Border  12'd8
  `define H_Front_Porch   12'd8
  `define H_Sync_Time     12'd96
  `define H_Back_Porch    12'd40
  `define H_Left_Border   12'd8

  `define V_Total_Time    12'd525
  `define V_Bottom_Border 12'd8
  `define V_Front_Porch   12'd2
  `define V_Sync_Time     12'd2
  `define V_Back_Porch    12'd25
  `define V_Top_Border    12'd8

`elsif Resolution_800x480
  `define H_Total_Time    12'd1056
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd40
  `define H_Sync_Time     12'd128
  `define H_Back_Porch    12'd88
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd525
  `define V_Bottom_Border 12'd8
  `define V_Front_Porch   12'd2
  `define V_Sync_Time     12'd2
  `define V_Back_Porch    12'd25
  `define V_Top_Border    12'd8

`elsif Resolution_800x600
  `define H_Total_Time    12'd1056
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd40
  `define H_Sync_Time     12'd128
  `define H_Back_Porch    12'd88
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd628
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd1
  `define V_Sync_Time     12'd4
  `define V_Back_Porch    12'd23
  `define V_Top_Border    12'd0

`elsif Resolution_1024x600
  `define H_Total_Time    12'd1344
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd24
  `define H_Sync_Time     12'd136
  `define H_Back_Porch    12'd160
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd628
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd1
  `define V_Sync_Time     12'd4
  `define V_Back_Porch    12'd23
  `define V_Top_Border    12'd0

`elsif Resolution_1024x768
  `define H_Total_Time    12'd1344
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd24
  `define H_Sync_Time     12'd136
  `define H_Back_Porch    12'd160
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd806
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd3
  `define V_Sync_Time     12'd6
  `define V_Back_Porch    12'd29
  `define V_Top_Border    12'd0

`elsif Resolution_1280x720
  `define H_Total_Time    12'd1650
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd110
  `define H_Sync_Time     12'd40
  `define H_Back_Porch    12'd220
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd750
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd5
  `define V_Sync_Time     12'd5
  `define V_Back_Porch    12'd20
  `define V_Top_Border    12'd0

`elsif Resolution_1920x1080
  `define H_Total_Time    12'd2200
  `define H_Right_Border  12'd0
  `define H_Front_Porch   12'd88
  `define H_Sync_Time     12'd44
  `define H_Back_Porch    12'd148
  `define H_Left_Border   12'd0

  `define V_Total_Time    12'd1125
  `define V_Bottom_Border 12'd0
  `define V_Front_Porch   12'd4
  `define V_Sync_Time     12'd5
  `define V_Back_Porch    12'd36
  `define V_Top_Border    12'd0

`endif

Top-Level Instantiation

Rom_Image_Tft.v

module Rom_Image_Tft (

  input clk50M,   // System clock input 50M
  input reset_n,  // Reset signal input, active low

  output [15:0] TFT_rgb,  // TFT data output
  output TFT_hs,         // TFT horizontal synchronization signal
  output TFT_vs,         // TFT vertical synchronization signal
  output TFT_clk,        // TFT pixel clock
  output TFT_de,         // TFT data enable
  output TFT_pwm         // TFT backlight control
);

  // Set the size of the image to be displayed, the address width of the ROM storing the image, and the background color of the display
  parameter DISP_IMAGE_W = 169;
  parameter DISP_IMAGE_H = 267;
  parameter ROM_ADDR_WIDTH = 16;     // Determined by the depth of the ROM bearing the image storage
  parameter DISP_BACK_COLOR = 16'hFFFF;  // White

  parameter TFT_WIDTH = 800;
  parameter TFT_HEIGHT = 480;

  // The image is displayed in the middle of the screen
  parameter DISP_HBEGIN = (TFT_WIDTH - DISP_IMAGE_W) / 2;
  parameter DISP_VBEGIN = (TFT_HEIGHT - DISP_IMAGE_H) / 2;

  wire pll_locked;
  wire clk_ctrl;
  wire tft_reset_n;
  wire [ROM_ADDR_WIDTH - 1:0] rom_addra;
  wire [15:0] disp_data;
  wire [15:0] rom_data;
  wire disp_data_req;
  wire [11:0] visible_hcount;
  wire [11:0] visible_vcount;
  wire frame_begin;
  wire tft_reset_p;
  wire [4:0] Disp_red;
  wire [5:0] Disp_green;
  wire [4:0] Disp_blue;
  wire clk33M;
  wire clk165M;

  assign tft_reset_n = pll_locked;
  assign tft_reset_p = ~pll_locked;
  assign clk_ctrl = clk33M;

  pll pll (
    // Clock out ports
    .clk_out1(clk33M),      // output clk_out1
    .clk_out2(clk165M),     // output clk_out2
    // Status and control signals
    .resetn(reset_n),     // input reset, active low
    .locked(pll_locked),  // output locked
    // Clock in ports
    .clk_in1(clk50M)       // input clk_in1
  );

  rom_image rom_image (
    .clka(clk_ctrl),    // input wire clka
    .addra(rom_addra),  // input wire [16 : 0] addra
    .douta(rom_data)    // output wire [15 : 0] douta
  );

  Image_Extract
  #(
    .H_Visible_area(TFT_WIDTH),       // Screen display area width
    .V_Visible_area(TFT_HEIGHT),      // Screen display area height
    .IMG_WIDTH(DISP_IMAGE_W),        // Image width
    .IMG_HEIGHT(DISP_IMAGE_H),       // Image height
    .IMG_DATA_WIDTH(16),              // Image pixel width
    .ROM_ADDR_WIDTH(ROM_ADDR_WIDTH)  // Address width of the ROM storing the image
  )
  image_extract (
    .clk_ctrl(clk_ctrl),
    .reset_n(tft_reset_n),
    .img_disp_hbegin(DISP_HBEGIN),
    .img_disp_vbegin(DISP_VBEGIN),
    .disp_back_color(DISP_BACK_COLOR),
    .rom_addra(rom_addra),
    .rom_data(rom_data),
    .frame_begin(frame_begin),
    .disp_data_req(disp_data_req),
    .visible_hcount(visible_hcount),
    .visible_vcount(visible_vcount),
    .disp_data(disp_data)
  );

  Disp_Driver disp_driver (
    .clk_disp(clk_ctrl),
    .rst_p(tft_reset_p),

    .Data_In(disp_data),
    .DataReq(disp_data_req),

    .H_Addr(visible_hcount),
    .V_Addr(visible_vcount),

    .Disp_Hs(TFT_hs),
    .Disp_Vs(TFT_vs),
    .Disp_Red(Disp_red),
    .Disp_Green(Disp_green),
    .Disp_Blue(Disp_blue),
    .Frame_Begin(frame_begin),  // Flag signal for the start of a frame image
    .Disp_De(TFT_de),
    .Disp_Pclk(TFT_clk)
  );

  assign TFT_rgb = {Disp_red, Disp_green, Disp_blue};
  assign TFT_pwm = 1'b1;

endmodule

TFT Display

Disp_Driver.v

`include "disp_param.vh"

module Disp_Driver (
  input clk_disp,
  input rst_p,

  input [`Red_Bits + `Green_Bits + `Blue_Bits - 1:0] Data_In,
  output DataReq,

  output [11:0] H_Addr,
  output [11:0] V_Addr,

  output reg Disp_Hs,
  output reg Disp_Vs,
  output reg [`Red_Bits - 1:0] Disp_Red,
  output reg [`Green_Bits - 1:0] Disp_Green,
  output reg [`Blue_Bits - 1:0] Disp_Blue,
  output reg Frame_Begin,  // Flag signal for the start of a frame image
  output reg Disp_De,
  output Disp_Pclk
);

  // Internal signals
  wire hcount_ov;  // Row counter overflow signal
  wire vcount_ov;  // Column counter overflow signal

  reg [11:0] hcount_r;  // Row counter register
  reg [11:0] vcount_r;  // Column counter register

  `ifdef HW_VGA
    assign Disp_Pclk = ~clk_disp;
  `else
    assign Disp_Pclk = clk_disp;
  `endif

  assign DataReq = Disp_De;

  // Timing parameters
  parameter Hdata_Begin = `H_Sync_Time + `H_Back_Porch + `H_Left_Border - 1'b1,
            Hdata_End = `H_Total_Time - `H_Right_Border - `H_Front_Porch - 1'b1,
            Vdata_Begin = `V_Sync_Time + `V_Back_Porch + `V_Top_Border - 1'b1,
            Vdata_End = `V_Total_Time - `V_Bottom_Border - `V_Front_Porch - 1'b1,
            VGA_HS_End = `H_Sync_Time - 1'b1,
            VGA_VS_End = `V_Sync_Time - 1'b1,
            Hpixel_End = `H_Total_Time - 1'b1,
            Vline_End = `V_Total_Time - 1'b1;

  // Row counter overflow signal
  assign hcount_ov = (hcount_r >= Hpixel_End);  // Row counter reaches maximum value, overflow signal is valid

  // Row counter
  always @(posedge clk_disp or posedge rst_p) begin
    if (rst_p)            // Reset, row counter cleared
      hcount_r <= 0;
    else if (hcount_ov)   // Reaches the end of the row scan, row counter cleared
      hcount_r <= 0;
    else                  // Otherwise, row counter increments by 1
      hcount_r <= hcount_r + 1;
  end

  // Column counter overflow signal
  assign vcount_ov = (vcount_r >= Vline_End);  // Column counter reaches maximum value and row counter overflows, overflow signal is valid

  // Column counter
  always @(posedge clk_disp or posedge rst_p) begin
    if (rst_p)          // Reset, column counter cleared
      vcount_r <= 0;
    else if (hcount_ov) begin  // Row counter overflows, column counter increments
      if (vcount_ov)    // Reaches the end of the column scan, column counter cleared
        vcount_r <= 0;
      else              // Otherwise, column counter increments by 1
        vcount_r <= vcount_r + 1;
    end
    else
      vcount_r <= vcount_r;
  end

  // Data valid area flag
  always @(posedge clk_disp) begin
    Disp_De <= ((hcount_r >= Hdata_Begin) && (hcount_r < Hdata_End)) &&
               ((vcount_r >= Vdata_Begin) && (vcount_r < Vdata_End));  // When both row and column counters are in the valid data area, the data valid area flag is valid
  end

  assign H_Addr = Disp_De ? (hcount_r - Hdata_Begin) : 12'd0;
  assign V_Addr = Disp_De ? (vcount_r - Vdata_Begin) : 12'd0;

  always @(posedge clk_disp) begin
    Disp_Hs <= (hcount_r >= VGA_HS_End);  // Row counter greater than or equal to VGS_HS_End, horizontal sync signal is valid
    Disp_Vs <= (vcount_r >= VGA_VS_End);  // Column counter greater than or equal to VGS_VS_End, vertical sync signal is valid
    {Disp_Red, Disp_Green, Disp_Blue} <= (Disp_De) ? Data_In : 0;  // Output RGB data
  end

    // in the valid data area, otherwise output black
  end

  // Extract rising edge of Disp_Vs
  reg Disp_Vs_Dly1;
  always @(posedge clk_disp) begin
    Disp_Vs_Dly1 <= Disp_Vs;
  end

  always @(posedge clk_disp or posedge rst_p) begin
    if (rst_p) begin
      Frame_Begin <= 0;
    end else if (!Disp_Vs_Dly1 && Disp_Vs) begin
      Frame_Begin <= 1;
    end else begin
      Frame_Begin <= 0;
    end
  end

endmodule

Result Display

HDMI Encoder

张智豪 — Thu, 06 Feb 2025 08:20:19 +0000

In the previous blog post, we introduced the HDMI encoding section. In this blog post, we will complete the subsequent task of HDMI: serial transmission of HDMI.

ODDR

Clock Selection

In the transmission stage, we usually use a clock frequency that is 5 times the encoder's working frequency to send the encoded 10-bit data. The reason is simple: when using DDR (Double Data Rate, where both the rising and falling edges of the clock are effective), under the clk_5x frequency, the time to serially transmit 10 bits of data is roughly the same as the time for the encoding part to process 10 bits of data under clk, which is beneficial for achieving data synchronization.

Template Usage

In FPGAs, we can use ready-made DDR templates to accomplish this.

Click on the Tool bar, find Language Templates.

In Verilog's Device Primitive Instantiation, find the Artixe-7 series; click on I/O Components, and find DDR Registers.

We need to serially output the encoded data, so we use ODDR (Output DDR Register); select the ODDR column, and you can see the preview of the template code on the left. Copy it as needed.

Note: You need to select the template library according to the series of FPGA chips you are using. Do not blindly follow the example.

Port Introduction

The ODDR structure diagram is as follows:

The port descriptions are as follows:

Port Signal Name	I/O	Bit Width	Description
C	I	1	Input Clock
CE	I	1	Clock Enable
D1	I	1	High Segment Input
D2	I	1	Low Segment Input
S/R	I	1	Set/Reset
Q	O	1	Output

Here, we explain the two working modes of ODDR: Opposite Edge and Same Edge.

In Opposite Edge mode, input data is latched on the rising edge of the clock signal and output on the falling edge; simultaneously, the next input data is latched on the falling edge of the clock signal and output on the next rising edge. In this mode, the output transmission delay is low, and data recovery is easier, but the power consumption is higher, and the quality requirement for the clock is also very high.

In Same Edge mode, input data is latched simultaneously on either the rising or falling edge of the clock signal, and output on both the rising and falling edges**. In this mode, the two edges of the output data correspond to the same input data. This method can simplify timing design and facilitate control. It has low requirements for the clock and lower power consumption, but the delay is higher.

Correspondence between DDR Interface and TMDS Data

We use the ODDR core in FPGA to transmit TMDS data.

HDMI has a total of 4 channels, of which 3 channels are data channels and one channel is a clock channel. Each channel needs to complete 10-bit data output within 5 clock cycles. Since the output data starts from the low bit, it is necessary to switch between datain_h and datain_1 every clock cycle. The specific correspondence is as follows:

OBUFDS

Since the TMDS encoding method uses differential transmission during transmission, that is, two signal lines are used for each channel, and their transmission levels are exactly opposite. To achieve this function, we can use the OBUFDS template library. The usage method is the same as ODDR, so we will not repeat it here.

Code

Serial Transmission Section

`timescale 1ns / 1ps

module Ser_Def_10to1(
    input clk_x5, // 5x clock frequency
    input [9:0] datain_0, // Channel 0 data input
    input [9:0] datain_1, // Channel 1 data input
    input [9:0] datain_2, // Channel 2 data input
    input [9:0] datain_3, // Channel 3 data input

    output wire dataout_0_n, // Channel 0 differential negative output
    output wire dataout_0_p, // Channel 0 differential positive output
    output wire dataout_1_n, // Channel 1 differential negative output
    output wire dataout_1_p, // Channel 1 differential positive output
    output wire dataout_2_n, // Channel 2 differential negative output
    output wire dataout_2_p, // Channel 2 differential positive output
    output wire dataout_3_n, // Channel 3 differential negative output
    output wire dataout_3_p  // Channel 3 differential positive output
);

// modulo-5 counter, used to select data bits
reg[2:0]TMDS_CNT5=0;

// shift registers
reg[4:0]TMDS_Shift_0h=0, TMDS_Shift_01=0;
reg[4:0]TMDS_Shift_1h=0, TMDS_Shift_11=0;
reg[4:0]TMDS_Shift_2h=0, TMDS_Shift_21=0;
reg[4:0]TMDS_Shift_3h=0, TMDS_Shift_31=0;

wire [4:0] TMDS_0_1 = {datain_0[9],datain_0[7],datain_0[5],datain_0[3],datain_0[1]};
wire [4:0] TMDS_0_h = {datain_0[8],datain_0[6],datain_0[4],datain_0[2],datain_0[0]};

wire [4:0] TMDS_1_1 = {datain_1[9],datain_1[7],datain_1[5],datain_1[3],datain_1[1]};
wire [4:0] TMDS_1_h = {datain_1[8],datain_1[6],datain_1[4],datain_1[2],datain_1[0]};

wire [4:0] TMDS_2_1 = {datain_2[9],datain_2[7],datain_2[5],datain_2[3],datain_2[1]};
wire [4:0] TMDS_2_h = {datain_2[8],datain_2[6],datain_2[4],datain_2[2],datain_2[0]};

wire [4:0] TMDS_3_1 = {datain_3[9],datain_3[7],datain_3[5],datain_3[3],datain_3[1]};
wire [4:0] TMDS_3_h = {datain_3[8],datain_3[6],datain_3[4],datain_3[2],datain_3[0]};


// 5x speed to send data
always @(posedge clk_x5) begin
    // TMDS_CNT5[2] is 1 for the first time, just count to 100, which is 4, 0 to 4 -->> 5 numbers are counted
    TMDS_CNT5 <= (TMDS_CNT5[2]) ? 3'd0 : TMDS_CNT5 + 3'd1;
    TMDS_Shift_0h <= (TMDS_CNT5[2]) ? TMDS_0_h : TMDS_Shift_0h[4:1];
    TMDS_Shift_01 <= (TMDS_CNT5[2]) ? TMDS_0_1 : TMDS_Shift_01[4:1];
    TMDS_Shift_1h <= (TMDS_CNT5[2]) ? TMDS_1_h : TMDS_Shift_1h[4:1];
    TMDS_Shift_11 <= (TMDS_CNT5[2]) ? TMDS_1_1 : TMDS_Shift_11[4:1];
    TMDS_Shift_2h <= (TMDS_CNT5[2]) ? TMDS_2_h : TMDS_Shift_2h[4:1];
    TMDS_Shift_21 <= (TMDS_CNT5[2]) ? TMDS_2_1 : TMDS_Shift_21[4:1];
    TMDS_Shift_3h <= (TMDS_CNT5[2]) ? TMDS_3_h : TMDS_Shift_3h[4:1];
    TMDS_Shift_31 <= (TMDS_CNT5[2]) ? TMDS_3_1 : TMDS_Shift_31[4:1];
end


wire dataout_0, data_out1, data_out2, dataout_3;


// Use ODDR and OBUFDS
// Channel 0
ODDR #(
    .DDR_CLK_EDGE("OPPOSITE_EDGE"), // "OPPOSITE_EDGE" or "SAME_EDGE"
    .INIT(1'b0),                     // Initial value of Q: 1'b0 or 1'b1
    .SRTYPE("SYNC")                    // Set/Reset type: "SYNC" or "ASYNC"
) ODDR_0 (
    .Q(dataout_0),        // 1-bit DDR output
    .C(clk_x5),          // 1-bit clock input
    .CE(1'b1),         // 1-bit clock enable input
    .D1(TMDS_Shift_01[0]), // 1-bit data input (positive edge)
    .D2(TMDS_Shift_0h[0]), // 1-bit data input (negative edge)
    .R(1'b0),          // 1-bit reset
    .S(1'b0)           // 1-bit set
);

OBUFDS #(
    .IOSTANDARD("DEFAULT"), // Specify the output I/O standard
    .SLEW("SLOW")         // Specify the output slew rate
) OBUFDS_0 (
    .O(dataout_0_p),  // Diff_p output (connect directly to top-level port)
    .OB(dataout_0_n), // Diff_n output (connect directly to top-level port)
    .I(dataout_0)     // Buffer input
);

// Channel 1
ODDR #(
    .DDR_CLK_EDGE("OPPOSITE_EDGE"), // "OPPOSITE_EDGE" or "SAME_EDGE"
    .INIT(1'b0),                     // Initial value of Q: 1'b0 or 1'b1
    .SRTYPE("SYNC")                    // Set/Reset type: "SYNC" or "ASYNC"
) ODDR_1 (
    .Q(dataout_1),        // 1-bit DDR output
    .C(clk_x5),          // 1-bit clock input
    .CE(1'b1),         // 1-bit clock enable input
    .D1(TMDS_Shift_11[0]), // 1-bit data input (positive edge)
    .D2(TMDS_Shift_1h[0]), // 1-bit data input (negative edge)
    .R(1'b0),          // 1-bit reset
    .S(1'b0)           // 1-bit set
);

OBUFDS #(
    .IOSTANDARD("DEFAULT"), // Specify the output I/O standard
    .SLEW("SLOW")         // Specify the output slew rate
) OBUFDS_1 (
    .O(dataout_1_p),  // Diff_p output (connect directly to top-level port)
    .OB(dataout_1_n), // Diff_n output (connect directly to top-level port)
    .I(dataout_1)     // Buffer input
);


// Channel 2
ODDR #(
    .DDR_CLK_EDGE("OPPOSITE_EDGE"), // "OPPOSITE_EDGE" or "SAME_EDGE"
    .INIT(1'b0),                     // Initial value of Q: 1'b0 or 1'b1
    .SRTYPE("SYNC")                    // Set/Reset type: "SYNC" or "ASYNC"
) ODDR_2 (
    .Q(dataout_2),        // 1-bit DDR output
    .C(clk_x5),          // 1-bit clock input
    .CE(1'b1),         // 1-bit clock enable input
    .D1(TMDS_Shift_21[0]), // 1-bit data input (positive edge)
    .D2(TMDS_Shift_2h[0]), // 1-bit data input (negative edge)
    .R(1'b0),          // 1-bit reset
    .S(1'b0)           // 1-bit set
);

OBUFDS #(
    .IOSTANDARD("DEFAULT"), // Specify the output I/O standard
    .SLEW("SLOW")         // Specify the output slew rate
) OBUFDS_2 (
    .O(dataout_2_p),  // Diff_p output (connect directly to top-level port)
    .OB(dataout_2_n), // Diff_n output (connect directly to top-level port)
    .I(dataout_2)     // Buffer input
);

// Channel 3
ODDR #(
    .DDR_CLK_EDGE("OPPOSITE_EDGE"), // "OPPOSITE_EDGE" or "SAME_EDGE"
    .INIT(1'b0),                     // Initial value of Q: 1'b0 or 1'b1
    .SRTYPE("SYNC")                    // Set/Reset type: "SYNC" or "ASYNC"
) ODDR_3 (
    .Q(dataout_3),        // 1-bit DDR output
    .C(clk_x5),          // 1-bit clock input
    .CE(1'b1),         // 1-bit clock enable input
    .D1(TMDS_Shift_31[0]), // 1-bit data input (positive edge)
    .D2(TMDS_Shift_3h[0]), // 1-bit data input (negative edge)
    .R(1'b0),          // 1-bit reset
    .S(1'b0)           // 1-bit set
);

OBUFDS #(
    .IOSTANDARD("DEFAULT"), // Specify the output I/O standard
    .SLEW("SLOW")         // Specify the output slew rate
) OBUFDS_3 (
    .O(dataout_3_p),  // Diff_p output (connect directly to top-level port)
    .OB(dataout_3_n), // Diff_n output (connect directly to top-level port)
    .I(dataout_3)     // Buffer input
);

endmodule

HDMI Transmitter

As long as the encoder is instantiated and linked with image data stream data and control signals according to the HDMI protocol, the transmitter module can be implemented.

When TMDS sends data, it also sends the pixel clock (which we记为 tmds_clk) together. tmds_clk should be consistent with the pixel clock pixelclk frequency, not pixelclk_x5. tmds_clk is used to synchronize a complete encoded data, not a single data bit.

There are two ways to generate tmds_clk: one is to invert pixelclk; the other is to use ODDR to generate it in the form of data. We use ODDR to generate it in the form of data. This method can maintain the phase relationship between the output clock and the output data to a certain extent, which is convenient for the receiving end to synchronize. The method of generating the waveform is very simple: we assign 10'b11111_00000 to the input segment of ODDR, so that in the first 2.5 clock cycles, the output level is high, and in the latter 2.5 cycles, the output level is low.

`timescale 1ns / 1ps

module HDMI_Encoder(
    // Input global control signals
    input pixelclk,     // Pixel clock
    input pixelclk_x5,   // 5x pixel clock
    input rst_p,        // Reset, positive polarity

    // Input display data
    input [7:0]blue_din,  // Blue data input
    input [7:0]green_din, // Green data input
    input [7:0]red_din,   // Red data input

    // Input display control signals
    input hsync,        // Horizontal sync
    input vsync,        // Vertical sync
    input de,           // Data enable

    // Output
    output tdms_clk_p,   // TMDS clock positive output
    output tdms_clk_n,   // TMDS clock negative output
    output [2:0] tmds_data_p, // TMDS data positive output
    output [2:0] tmds_data_n  // TMDS data negative output
);

wire [9:0]red;
wire [9:0]green;
wire [9:0]blue;

// Instantiate modules
Encode encb(
    .clk(pixelclk),     // Pixel clock
    .rst_n(~rst_p),    // Reset, negative polarity
    .din(blue_din),    // Data input, blue
    .c0(hsync),       // Control signal c0, horizontal sync
    .c1(vsync),       // Control signal c1, vertical sync
    .de(de),          // Data enable
    .dout(blue)       // Data output, blue
);

Encode encg(
    .clk(pixelclk),     // Pixel clock
    .rst_n(~rst_p),    // Reset, negative polarity
    .din(green_din),   // Data input, green
    .c0(1'b0),         // Control signal c0, constant 0
    .c1(1'b0),         // Control signal c1, constant 0
    .de(de),          // Data enable
    .dout(green)      // Data output, green
);

Encode encr(
    .clk(pixelclk),     // Pixel clock
    .rst_n(~rst_p),    // Reset, negative polarity
    .din(red_din),     // Data input, red
    .c0(1'b0),         // Control signal c0, constant 0
    .c1(1'b0),         // Control signal c1, constant 0
    .de(de),          // Data enable
    .dout(red)        // Data output, red
);

Ser_Def_10to1 HDMI_Sender(
    .clk_x5(pixelclk_x5),     // 5x pixel clock
    .datain_0(blue),        // Data input channel 0, blue
    .datain_1(green),       // Data input channel 1, green
    .datain_2(red),         // Data input channel 2, red
    .datain_3(10'b11111_00000), // Data input channel 3, for clock generation
    .dataout_0_n(tmds_data_p[0]), // Data output channel 0 negative, positive polarity
    .dataout_0_p(tmds_data_n[0]), // Data output channel 0 positive, negative polarity
    .dataout_1_n(tmds_data_p[1]), // Data output channel 1 negative, positive polarity
    .dataout_1_p(tmds_data_n[1]), // Data output channel 1 positive, negative polarity
    .dataout_2_n(tmds_data_p[2]), // Data output channel 2 negative, positive polarity
    .dataout_2_p(tmds_data_n[2]), // Data output channel 2 positive, negative polarity
    .dataout_3_n(tmds_clk_n),    // Clock output negative, negative polarity
    .dataout_3_p(tmds_clk_p)     // Clock output positive, positive polarity
);

endmodule

Note: The Encode module code can be found in my previous article

After completing this controller, when using it, you only need to connect the corresponding signals to the HDMI_Encoder corresponding ports based on the VGA or TFT display system.

Top Level

`timescale 1ns / 1ps

module HDMI_Disp(
    input clk_50M,       // 50MHz system clock
    input rst_n,         // Reset, negative polarity

    // HDMI1 interface
    output hdmi1_clk_p,   // HDMI1 clock positive output
    output hdmi1_clk_n,   // HDMI1 clock negative output
    output [2:0]hdmi1_dat_p, // HDMI1 data positive output
    output [2:0]hdmi1_data_n, // HDMI1 data negative output
    output [2:0]hdmi1_oe,   // HDMI1 output enable

    // HDMI2 interface
    output hdmi2_clk_p,   // HDMI2 clock positive output
    output hdmi2_clk_n,   // HDMI2 clock negative output
    output [2:0]hdmi2_dat_p, // HDMI2 data positive output
    output [2:0]hdmi2_data_n, // HDMI2 data negative output
    output [2:0]hdmi2_oe,   // HDMI2 output enable

    // HDMI3 interface
    output hdmi3_clk_p,   // HDMI3 clock positive output
    output hdmi3_clk_n,   // HDMI3 clock negative output
    output [2:0]hdmi3_dat_p, // HDMI3 data positive output
    output [2:0]hdmi3_data_n, // HDMI3 data negative output
    output [2:0]hdmi3_oe,   // HDMI3 output enable

    // TFT interface
    output [15:0]TFT_rgb, // TFT RGB data output
    output TFT_hs,       // TFT horizontal sync
    output TFT_vs,       // TFT vertical sync
    output TFT_clk,      // TFT clock
    output TFT_de,       // TFT data enable
    output TFT_pwn       // TFT PWM
);

// Resolution_800x480 // Clock is 33MHz
parameter
    Disp_Width = 800,
    Disp_Height = 480;

wire pixelclk;      // Pixel clock, 33MHz
wire pixelclk_x5;    // 5x pixel clock, 165MHz
wire pll_locked;    // PLL locked signal

wire rst_p;         // Reset, positive polarity
wire [11:0] disp_h_addr; // Display horizontal address
wire [11:0] disp_v_addr; // Display vertical address
wire disp_data_req;  // Display data request
wire [23:0]disp_data;   // Display data, 24 bits RGB
wire disp_hs;        // Display horizontal sync
wire disp_vs;        // Display vertical sync
wire [7:0]disp_red;    // Display red data
wire [7:0]disp_green;  // Display green data
wire [7:0]disp_blue;   // Display blue data
wire disp_de;        // Display data enable
wire disp_pclk;      // Display pixel clock

assign rst_p = ~pll_locked;


pll pll (
    .clk_out1(pixelclk),   // output 33M
    .clk_out2(pixelclk_x5),  // output 165M
    .resetn(rst_n),      // input resetn
    .locked(pll_locked),   // output locked
    .clk_in1(clk_50M)    // input clk_in1
);

Color_Bar #(
    .Disp_Width(Disp_Width),
    .Disp_Height(Disp_Height)
) color_bar (
    .disp_h_addr(disp_h_addr),
    .disp_v_addr(disp_v_addr),
    .disp_data_req(disp_data_req),
    .disp_data(disp_data)
);

Disp_Driver disp_driver (
    .clk_disp (pixelclk),
    .rst_p (rst_p),
    .Data_In (disp_data),
    .DataReq (disp_data_req),
    .H_Addr (disp_h_addr),
    .V_Addr (disp_v_addr),
    .Disp_Hs (disp_hs),
    .Disp_Vs (disp_vs),
    .Disp_Red (disp_red),
    .Disp_Green (disp_green),
    .Disp_Blue (disp_blue),
    .Frame_Begin (),       // Flag for the beginning of a frame
    .Disp_De (disp_de),
    .Disp_Pclk (disp_pclk)
);

// TFT
assign TFT_rgb = {disp_red[7:3], disp_green[7:2], disp_blue[7:3]};
assign TFT_hs = disp_hs;
assign TFT_vs = disp_vs;
assign TFT_clk = disp_pclk;
assign TFT_de = disp_de;
assign TFT_pwm = 1'b1;

HDMI_Encoder encoder1 (
    // Input global control signals
    .pixelclk (pixelclk),
    .pixelclk_x5 (pixelclk_x5),
    .rst_p (rst_p),
    .blue_din (disp_blue),
    .green_din (disp_green),
    .red_din (disp_red),
    .hsync (disp_hs),
    .vsync (disp_vs),
    .de (disp_de),
    .tdms_clk_p (hdmi1_clk_p),
    .tdms_clk_n (hdmi1_clk_n),
    .tmds_data_p (hdmi1_dat_p),
    .tmds_data_n (hdmi1_data_n)
);

assign hdmi1_oe = 1'b1;

HDMI_Encoder encoder2 (
    // Input global control signals
    .pixelclk (pixelclk),
    .pixelclk_x5 (pixelclk_x5),
    .rst_p (rst_p),
    .blue_din (disp_blue),
    .green_din (disp_green),
    .red_din (disp_red),
    .hsync (disp_hs),
    .vsync (disp_vs),
    .de (disp_de),
    .tdms_clk_p (hdmi2_clk_p),
    .tdms_clk_n (hdmi2_clk_n),
    .tmds_data_p (hdmi2_dat_p),
    .tmds_data_n (hdmi2_data_n)
);

assign hdmi2_oe = 1'b1;


endmodule

Auxiliary Modules

Color_Bar

`timescale 1ns / 1ps

module Color_Bar #(
    parameter Disp_Width = 800,
    parameter Disp_Height = 480
)(
    input [11:0]disp_h_addr,   // Display horizontal address input
    input [11:0]disp_v_addr,   // Display vertical address input
    input disp_data_req,     // Display data request input
    output reg [23:0] disp_data // Display data output
);

localparam
    BLACK    = 24'h000000, // Black color
    BLUE     = 24'h0000FF, // Blue color
    RED      = 24'hFF0000, // Red color
    PURPPLE  = 24'hFF00FF, // Purple color
    GREEN    = 24'h00FF00, // Green color
    CYAN     = 24'h00FFFF, // Cyan color
    YELLOW   = 24'hFFFF00, // Yellow color
    WHITE    = 24'hFFFFFF; // White color

// Define the default display color value for each pixel block
localparam
    R0_C0 = BLACK,    // Pixel block in row 0, column 0
    R0_C1 = BLUE,     // Pixel block in row 0, column 1
    R1_C0 = RED,      // Pixel block in row 1, column 0
    R1_C1 = PURPPLE,  // Pixel block in row 1, column 1
    R2_C0 = GREEN,    // Pixel block in row 2, column 0
    R2_C1 = CYAN,     // Pixel block in row 2, column 1
    R3_C0 = YELLOW,   // Pixel block in row 3, column 0
    R3_C1 = WHITE;    // Pixel block in row 3, column 1

wire [3:0]row_act;   // Active row flags
wire [1:0]col_act;   // Active column flags

genvar i, j;
localparam row_height = Disp_Height / 4, // Height of each row block
           col_width  = Disp_Width / 2;  // Width of each column block
for (i = 0; i < 4; i = i + 1) begin
    assign row_act[i] = (disp_v_addr >= i * row_height) && (disp_v_addr < (1 + i) * row_height);
end
for (j = 0; j < 2; j = j + 1) begin
    assign col_act[j] = (disp_h_addr >= j * col_width) && (disp_h_addr < (1 + j) * col_width);
end

always @(*)
    case ({row_act, col_act, disp_data_req})
        7'b0001_01_1: disp_data = R0_C0;
        7'b0001_10_1: disp_data = R0_C1;
        7'b0010_01_1: disp_data = R1_C0;
        7'b0010_10_1: disp_data = R1_C1;
        7'b0100_01_1: disp_data = R2_C0;
        7'b0100_10_1: disp_data = R2_C1;
        7'b1000_01_1: disp_data = R3_C0;
        7'b1000_10_1: disp_data = R3_C1;
        default:      disp_data = R0_C0;
    endcase

endmodule

Header File

disp_para.vh

/* Usage instructions
When using, select 2 predefined parameters according to actual work needs.

Parameter 1: MODE_RGBxxx
Predefined to determine whether the driver works in 16-bit mode or 24-bit mode, choose one of the two
MODE_RGB888: 24-bit mode
MODE_RGB565: 16-bit mode
4.3-inch TFT display ------ use 16-bit color RGB565 mode
5-inch TFT display -------- use 16-bit color RGB565 mode
GM7123 module ---------- use 24-bit color RGB888 mode

Parameter 2: Resolution_xxxx
Predefined to determine the resolution of the display device, common device resolutions are as follows

4.3-inch TFT display:
Resolution_480x272

5-inch TFT display:
Resolution_800x480

VGA common resolution:
Resolution_640x480
Resolution_800x600
Resolution_1024x600
Resolution_1024x768
Resolution_1280x720
Resolution_1920x1080
*/

// You can also set the display device type through macro definition. Enable one and shield others with comments.
// Use 4.3-inch 480*272 resolution display
//`define HW_TFT43

// Use 5-inch 800*480 resolution display
`define HW_TFT50

// Use VGA monitor, default is 640*480 resolution, 24-bit mode. Other resolutions or 16-bit mode can be reconfigured from line 63 to line 75 in the code.
//`define HW_VGA

//=====================================
// The following macro definitions are used to set the bit mode and resolution parameters according to the display device
//=====================================
`ifdef HW_TFT43 // Use 4.3-inch 480*272 resolution display
    `define MODE_RGB565
    `define Resolution_480x272 1 // Clock is 9MHz

`elsif HW_TFT50 // Use 5-inch 800*480 resolution display
    `define MODE_RGB565
    `define Resolution_800x480 1 // Clock is 33MHz

`elsif HW_VGA // Use VGA monitor, default is 640*480 resolution, 24-bit mode
    //=====================================
    // Other resolutions and 16-bit modes can be selected. Users need to set them according to actual needs.
    // Code lines 75~76 set the bit mode
    // Code lines 77~83 set the resolution
    //=====================================
    //`define MODE_RGB565
    `define MODE_RGB888
    //`define Resolution_640x480 1 // Clock is 25.175MHz
    //`define Resolution_800x480 1 // Clock is 33MHz, compatible with TFT5.0
    //`define Resolution_800x600 1 // Clock is 40MHz
    //`define Resolution_1024x600 1 // Clock is 51MHz
    //`define Resolution_1024x768 1 // Clock is 65MHz
    `define Resolution_1280x720 1 // Clock is 74.25MHz
    //`define Resolution_1920x1080 1 // Clock is 148.5MHz
`endif

//=====================================
// For non-special needs, users do not need to modify the following content
//=====================================
// Define different color depths
`ifdef MODE_RGB888
    `define Red_Bits 8
    `define Green_Bits 8
    `define Blue_Bits 8

`elsif MODE_RGB565
    `define Red_Bits 5
    `define Green_Bits 6
    `define Blue_Bits 5
`endif

// Define timing parameters for different resolutions
`ifdef Resolution_480x272
    `define H_Total_Time 12'd525
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd2
    `define H_Sync_Time 12'd41
    `define H_Back_Porch 12'd2
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd286
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd2
    `define V_Sync_Time 12'd10
    `define V_Back_Porch 12'd2
    `define V_Top_Border 12'd0

`elsif Resolution_640x480
    `define H_Total_Time 12'd800
    `define H_Right_Border 12'd8
    `define H_Front_Porch 12'd8
    `define H_Sync_Time 12'd96
    `define H_Back_Porch 12'd40
    `define H_Left_Border 12'd8

    `define V_Total_Time 12'd525
    `define V_Bottom_Border 12'd8
    `define V_Front_Porch 12'd2
    `define V_Sync_Time 12'd2
    `define V_Back_Porch 12'd25
    `define V_Top_Border 12'd8

`elsif Resolution_800x480
    `define H_Total_Time 12'd1056
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd40
    `define H_Sync_Time 12'd128
    `define H_Back_Porch 12'd88
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd525
    `define V_Bottom_Border 12'd8
    `define V_Front_Porch 12'd2
    `define V_Sync_Time 12'd2
    `define V_Back_Porch 12'd25
    `define V_Top_Border 12'd8

`elsif Resolution_800x600
    `define H_Total_Time 12'd1056
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd40
    `define H_Sync_Time 12'd128
    `define H_Back_Porch 12'd88
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd628
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd1
    `define V_Sync_Time 12'd4
    `define V_Back_Porch 12'd23
    `define V_Top_Border 12'd0

`elsif Resolution_1024x600
    `define H_Total_Time 12'd1344
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd24
    `define H_Sync_Time 12'd136
    `define H_Back_Porch 12'd160
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd628
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd1
    `define V_Sync_Time 12'd4
    `define V_Back_Porch 12'd23
    `define V_Top_Border 12'd0

`elsif Resolution_1024x768
    `define H_Total_Time 12'd1344
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd24
    `define H_Sync_Time 12'd136
    `define H_Back_Porch 12'd160
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd806
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd3
    `define V_Sync_Time 12'd6
    `define V_Back_Porch 12'd29
    `define V_Top_Border 12'd0

`elsif Resolution_1280x720
    `define H_Total_Time 12'd1650
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd110
    `define H_Sync_Time 12'd40
    `define H_Back_Porch 12'd220
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd750
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd5
    `define V_Sync_Time 12'd5
    `define V_Back_Porch 12'd20
    `define V_Top_Border 12'd0


`elsif Resolution_1920x1080
    `define H_Total_Time 12'd2200
    `define H_Right_Border 12'd0
    `define H_Front_Porch 12'd88
    `define H_Sync_Time 12'd44
    `define H_Back_Porch 12'd148
    `define H_Left_Border 12'd0

    `define V_Total_Time 12'd1125
    `define V_Bottom_Border 12'd0
    `define V_Front_Porch 12'd4
    `define V_Sync_Time 12'd5
    `define V_Back_Porch 12'd36
    `define V_Top_Border 12'd0

`endif

Beyond the Program: The Three Layers of Computer Abstraction and the Underlying Logic of Performance Optimization

张智豪 — Wed, 05 Feb 2025 02:52:35 +0000

Have you ever wondered how a computer understands the code you write? How does it transform high-level languages into instructions that the hardware can execute? Most people are only involved in calling functions but don't truly understand the underlying principles. This article will reveal what lies "beyond the program" and how we define computer performance.

Beyond the Program

The Three Layers of Abstraction

Have you ever considered what your computer actually is? Essentially, it's a collection of hardware components, a system that processes 0s and 1s. We can abstract your laptop into three layers: application software, system software, and hardware.

Application software is like a complex instruction manual written by programmers, but it's in a language that the computer can't directly understand.
System software acts as a bridge between the application software and the hardware, translating the complex manual into simple, easy-to-understand strings of 0s and 1s.
Hardware is responsible for executing tasks such as displaying content on the screen, playing music, and performing calculations. It's like a strong worker who doesn't make mistakes and is incredibly efficient, but you need to communicate in its language to tell it what to do and how to use its resources.

Which of these three layers is the most important? The answer varies depending on your perspective, but from a technological development standpoint, the outermost layer is the most likely to be impacted by AI. Interestingly, one of the reasons is that it's the easiest for us to get started with.

Application software is the most closely related to our daily lives, including games, social media applications, and search engines. System software is a bit less familiar, and many people are often confused when using it. For example, I still remember my first time using Git to upload files to GitHub. The language based on the operating system made it difficult for me to adapt at first.

Hardware, on the other hand, is like a "familiar stranger." We all use it (like mice, keyboards, and screens) and have heard of components like CPUs and GPUs, but do we understand their working principles? The answer is likely no. This is why people working in hardware are in high demand. People tend to underestimate hardware (like mice and keyboards) or consider applications simple (like CPUs and GPUs), leading them to focus solely on software development.

What about system software? The field of application software, with its established framework, has reached a point of saturation. Why? Let's delve deeper.

System software encompasses many components, with the two most critical being the operating system and the compiler. Their primary tasks include handling basic input and output operations, allocating external and internal memory, and providing shared computing resources for multiple applications. In other words, it serves as the interface between pure software and pure hardware. The operating systems we currently use, such as Linux, iOS, and Windows, are unlikely to be replaced by other operating systems, considering the immense amount of engineering required and the high level of difficulty. Apart from Huawei, no other company dares, wants, or has the capability to attempt such an endeavor.

The other key component is the compiler, which translates programs written in high-level languages (such as C, C++, Java, and Python) into instructions that the hardware can execute. This process is quite complex, and we'll provide a brief overview here, with a more detailed explanation in a future article.

From High-Level Language to Hardware Language

Computers can only understand binary data and, based on the bit segments of the instructions, extract the corresponding data and perform the corresponding operations (calculations, shifts, storage, and retrieval). For example, 1001010100101110 would instruct the computer to add two numbers. Using numbers to represent both instructions and data is the foundation of computing. I'll be writing a detailed article about instructions later on, so if you're interested, please like and follow me to stay updated.

Obviously, a string of numbers is tedious and lacks readability, which led to the development of assemblers. These are software programs that translate mnemonic instructions into corresponding binary code, instructing the computer to add two numbers A and B.

add A, B --> 1001010100101110

However, this still wasn't intuitive enough, so people created high-level programming languages like C++, C, and Java, which use portable languages composed of words and algebraic symbols. Compilers translate these into assembly language.

A + B --> add A, B

At this point, you might have a general idea of what happens. When we run code, our compiler checks the syntax and then translates the high-level language we've written into assembly language. With the help of the assembler, the assembly language is then translated into instructions that the computer can understand. These instructions are stored in memory, and during each clock cycle, the CPU fetches these instructions and performs the corresponding operations. The process is illustrated below:

Performance

When it comes to computers, everyone loves to talk about performance. So, how do we define performance? One way is to compare the execution time of the same program, similar to a race. This is the speed perspective, which is commonly used for CPUs. Another perspective is throughput, such as how many instructions can be executed in a clock cycle, which is the core idea behind GPUs. Still having trouble understanding? Imagine transporting 450 tourists from Shanghai to Beijing. There are several ways to do it. Using a plane or high-speed train might be sufficient for a single trip, while using small cars would require multiple trips back and forth.

Let's consider performance from the speed perspective:

Performance = 1 / Execution Time

Let's look at an example:

If computer A takes 10 seconds to run a program and computer B takes 15 seconds, how much faster is computer A than computer B?

Performance_A / Performance_B = Execution_Time_B / Execution_Time_A = 15 / 10 = 1.5

To be more precise, the execution time we consider is the CPU execution time, and we use clock cycles as the unit of measurement. So we have:

CPU Execution Time of a Program = CPU Clock Cycles of a Program * Clock Cycle Length

or CPU Execution Time of a Program = CPU Clock Cycles of a Program / Clock Frequency

Now, let's consider instructions:

Number of CPU Clock Cycles = Number of Program Instructions * Average Clock Cycles per Instruction (CPI)

CPI stands for "clock cycle per instruction."

Let's look at another example:

Suppose we have two different implementations of the same instruction set. Computer A has a clock cycle length of 250 ps and a CPI of 2.0 for a certain program. Computer B has a clock cycle length of 500 ps and a CPI of 1.2 for the same program. Which computer is faster for this program?

Number of clock cycles:

CPU Clock Cycles_A = I * 2.0

CPU Clock Cycles_B = I * 1.2

CPU time:

CPU Time_A = CPU Clock Cycles_A * Clock Cycle Length = I * 2.0 * 250 ps = 500 * I ps

CPU Time_B = CPU Clock Cycles_B * Clock Cycle Length = I * 1.2 * 500 ps = 600 * I ps

Performance:

CPU Performance_A / CPU Performance_B = CPU_B / CPU_A = 600 / 500 = 1.2

There's another way to express CPU time:

CPU Time = (Number of Instructions * CPI) / Clock Frequency

This method allows us to analyze the performance impact of each type of instruction. However, we won't delve into that here.

So, how do we improve computer performance? From the formula, we can see that the shorter the CPU time, the better the performance. Therefore, a faster clock frequency and fewer cycles required for program execution are what we desire. A faster clock frequency is related to hardware, while the number of clock cycles required is related to algorithms. In situations where we can't change the hardware, optimizing the code based on the computer architecture to improve performance is what sets apart computer experts from ordinary programmers. Here's an intuitive insight: accelerate frequently occurring events. A real-world example is high-speed elevators in office buildings.

Finally, I want to share an important law: Amdahl's Law. It's quite simple: you can't expect a proportional improvement in overall performance from a local improvement in one aspect of the computer. Let's take a simple example:

Suppose a program takes 100 seconds to run on a computer, with 80 seconds spent on multiplication operations. If we improve the speed of program execution by 5 times, how much should the speed of multiplication operations be improved?

Time after improvement = (Execution time affected by the improvement) / (Improvement factor) + (Execution time not affected)

Time after improvement = 80 / n + (100 - 80)

Since we want to improve the execution time by 5 times, the new execution time should be 20 seconds. Therefore:

20 = 80 / n + 20

0 = 80 / n

This means that the multiplication operations need to be infinitely faster than before, which is obviously impossible.

You must have heard of the concept of multi-core. Similarly, simply stacking processors won't lead to a proportional improvement in performance, because the communication protocols between devices will limit their speed. In other words, using multi-core to improve computer performance follows a pattern of diminishing returns. Stacking cores eventually leads to a slowdown in performance improvement.

I believe you're also familiar with the famous scaling law. In simple terms, it means that as long as you blindly add more hardware, performance will skyrocket. However, according to Amdahl's Law, the end of the scaling law is a "toothpaste-like" improvement in performance.

The recent DeepSeek AI is perhaps the most vivid example. They don't need a lot of Nvidia H100 GPUs to achieve performance comparable to OpenAI's O1 model in certain aspects. This is because they are not blindly stacking hardware; they are optimizing algorithms based on the computer architecture.

General Overview Of Computer Architecture

张智豪 — Sun, 26 Jan 2025 11:05:20 +0000

Traditional Computer Architecture

The earliest computers were designed to perform specialized tasks and lacked the ability to be reprogrammed. These are known as "Fixed Program Computers," and a simple calculator is a perfect example of this type of computer.

Later, "Stored Program Computers" were developed to handle a wider range of tasks and applications. Many modern computers are based on the stored-program concept introduced by John Von Neumann, where both program instructions and data are stored in the same memory.

The power of this concept lies in its flexibility. The source code for an editor program, its corresponding compiled machine code, the text the program is editing, and even the compiler that generated the machine code can all reside in the same memory. This has significant commercial implications, as computers can utilize ready-made software as long as they are compatible with an existing instruction set.

We can conceptually divide a computer into three primary components: memory, processor, and datapath. The processor, also known as the Central Processing Unit (CPU), is responsible for data flow and arithmetic operations. Memory provides instructions that dictate the computer's actions and stores the data to be processed, facilitating data flow between memory and the CPU. This is the fundamental operational principle of the John Von Neumann architecture.

You might also be familiar with the Harvard Architecture, which employs separate memories and buses for instructions and data, aiming for improved performance and suitability for general-purpose computing.

While the Harvard Architecture differs from the John Von Neumann Architecture in its handling of data flow, the underlying principles remain the same: a separation between memory and the CPU.

These architectures have served us well as semiconductor technology has progressed. However, in the era of Artificial Intelligence (AI), significant challenges have emerged.

Challenges Faced by Current Computer Architecture

1.Memory Wall

The "memory wall" refers to the growing disparity between processor speeds and memory bandwidth.

This increasing gap results in processors spending more time waiting for data from memory rather than performing computations, creating a significant performance bottleneck.

2.Power Wall

The "power wall" represents the escalating problem of heat dissipation in chips. As transistor density and clock speeds increase, the power consumption and resulting heat generation become more challenging to manage.

3.Specialization Vs Generalization Dilemma

Traditional architectures were designed for general-purpose computing. However, specialized workloads like AI often benefit from specialized hardware accelerators (e.g., GPUs, TPUs). Striking a balance between general-purpose capabilities and specialized acceleration poses a major challenge.

Possible Solutions

Processing-In-Memory: Integrating processing elements directly within the memory array itself, allowing computations to occur where the data resides.
Neuromorphic Computing: Inspired by the human brain, using asynchronous, event-driven computation for AI workloads.
Optical Computing: Using light instead of electricity for computation, offering potential advantages in speed and power efficiency.
Quantum Computing: While still in its nascent stages, quantum computing holds the potential to revolutionize computation by leveraging quantum mechanics to solve problems currently intractable for classical computers.

References

[1]https://medium.com/@kalebmlemke/the-importance-of-ai-accelerators-in-ai-development-c7012d5e175a

[2]https://www.geeksforgeeks.org/harvard-architecture

[3]https://www.geeksforgeeks.org/computer-organization-von-neumann-architecture

[4]https://ayarlabs.com/glossary/memory-wall

[5] Computer Organization and Design MIPS Edition Authors: David A. Patterson, John L. Hennessy

Fundamentals of FPGA

张智豪 — Sat, 18 Jan 2025 09:45:28 +0000

For some reason, there exists an implicit hierarchy of disdain in embedded development: those working on 8051 microcontrollers tend to look down on basic circuit developers, STM32 developers look down on those working with 8051, and FPGA developers look down on STM32 developers. Today, our main focus is on FPGA!

What is FPGA?

A Field Programmable Gate Array(FPGA) is a type of configurable integrated circuit which can be repeatedly programmed after manufacturing.

Many customers favor FPGA over MCU for its programmability and flexibility for the fact that customers could customize their own circuits based on their needs. In addition, FPGA boast higher performance because of its parallel structure, meaning they can process data in a given clock than MCU does. Consequently, FPGA chips are more expensive than MCUs.

How it works

Boolean Algebra

In logic, we often use true (1) or false (0) to judge whether an event occurs. For example, we use a variable A to denote whether it will rain tomorrow. If it rains, A equals 1, which means the event occurs; otherwise, A equals 0.

This logic is analogous to the use of high (1) and low (0) voltage levels in circuits to detect specific events and enable the circuit to respond accordingly.

To perform these logical operations, circuits rely on logic gates. The equivalent Boolean operations of logic gates are as follows:

Or Gate: Performs Boolean addition
And Gate: Performs Boolean multiplication

Or gate:

And Gate:

Sum of Products

Essentially, Or Gates (Boolean addition) and And Gates (Boolean multiplication) can be combined to detect multiple events. This is represented in the Sum of Products form, where a function is expressed as a series of AND operations (products) combined through OR operations (summation). For example:

F = (A AND B) OR (B AND C AND D) OR(A AND C).

Ideally, as long as there are sufficient resources, we can design circuits to represent the world in terms of 0s and 1s. This is the fundamental concept behind how computers work.

Truth Table

A Truth Table is a tabular representation of all possible input combinations and their corresponding outputs for a logic circuit or function. It is used to understand and verify the behavior of the circuit.

For example, consider a simple AND gate with two inputs, A and B, and one output, F. The truth table would look like this:

| A | B | F (A AND B) |
| ----- | ----- | --------------- |
| 0| 0| 0|
| 0| 1| 0|
| 1| 0| 0|
| 1| 1| 1|

The Truth Table not only helps in analyzing individual gates but also serves as a blueprint for designing and validating more complex functions implemented within an FPGA. When circuits become large and complex, tools like LUTs are used to handle these truth table mappings efficiently.

Lookup Tables (LUTs)

Lookup Tables (LUTs) are a direct hardware implementation of truth tables in an FPGA. They are small memory units that store the output values for all possible input combinations of a logic function. This means an LUT essentially maps the truth table into a programmable circuit.

For example, consider the same AND gate as above. The LUT for this gate would store the output column of the truth table:

| Address (Input Combination) | Stored Value (Output) |
| ------------------------------- | ------------------------- |
| 00 | 0 |
| 01 | 0 |
| 10 | 0 |
| 11 | 1 |

Here:

The address corresponds to the input combination (e.g., A=0, B=1 corresponds to 01 in binary).
The stored value represents the corresponding output for that combination.

The relationship between the truth table and LUTs allows FPGAs to implement logic functions efficiently:

Truth Table as the Blueprint: The truth table defines the desired behavior of the logic circuit.
LUT as the Hardware Realization: The LUT translates this behavior into a physical implementation by storing the outputs for all possible inputs.

By leveraging LUTs, FPGAs can dynamically reprogram their hardware to implement a wide variety of functions, making them highly versatile. For larger designs, multiple LUTs are interconnected through the FPGA’s programmable interconnects, enabling the implementation of complex logic circuits without the need for custom hardware.

Interconnects

Interconnects are the backbone of an FPGA's programmable wiring network. They connect Lookup Tables (LUTs), registers, and other logic elements, allowing the FPGA to form custom circuits based on the user’s design. Without interconnects, the individual LUTs would remain isolated and unable to contribute to a larger, functional design.

### Relationship Between Interconnects and LUTs

While LUTs implement the logic of individual truth tables, interconnects are responsible for connecting the outputs of one LUT to the inputs of another. This connectivity enables the FPGA to scale simple logic operations into complex systems.

For example:

A single LUT might implement a basic logic function like A AND B.
Through interconnects, the output of this LUT could be routed as an input to another LUT that performs a function like (A AND B) OR C.

The interconnect system in an FPGA consists of several programmable elements that allow signals to traverse the chip:

Global Routing Resources: These are long wires that span across large areas of the FPGA, enabling signals to travel between distant parts of the chip.
Switch Matrices: These are grids of programmable switches that control how signals move between different routing paths. Users configure these switches to create the connections needed for their design.
Local Routing Resources: Short wires are used to connect nearby logic elements, such as linking one LUT to a neighboring LUT.
Dedicated Lines: For high-speed or specialized connections, dedicated lines provide direct, efficient pathways that bypass general-purpose routing.