DEV Community: Ismiel Abir

Machine Learning Developmental Life Cycle

Ismiel Abir — Fri, 07 Jun 2024 05:23:48 +0000

For development any machine learning project you can follow below life cycle:

Identify/Frame the problem
Gather or collect data
Data preprocessing
Exploratory data analysis
Feature engineering and selection
Model training, evaluation and selection
Testing
Deployment

ValueError: A given column is not a column of the dataframe

Ismiel Abir — Thu, 06 Jun 2024 12:45:18 +0000

Sometimes, when attempting to use the 'pipeline,' I encounter an error called 'ValueError: A given column is not a column of the dataframe.' However, the column still exists in the dataset.

I found this error to be quite interesting. Initially, I thought about seeking help on Kaggle or Stack Overflow to solve this problem. Eventually, I discovered the solution on Stack Overflow, and now I understand why this error occurs. It's really interesting.

Let me describe how to overcome this error…



from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

preprocessor = ColumnTransformer(
    transformers=[
        ('ohe', OneHotEncoder(drop='first', sparse=False, handle_unknown='ignore'), ohe),
        ('scaler', StandardScaler(), scaler),
        ('ordinal', OrdinalEncoder(), ordinal),
        ('label', LabelEncoder(), label)
    ], remainder='passthrough')

I attempted to use 'ColumnTransformer' to preprocess everything, and that worked fine. However, when I tried to fit the pipeline, a 'column not found' error occurred.

The reason for this error is that I used the target value 'LabelEncoder' in the preprocessing section. However, the target/output part should be handled independently, as shown in the code below:



label = LabelEncoder()

y_label_train = label.fit_transform(y_train)
y_label_test = label.transform(y_test)

After that, we can fit this without errors.

If there is another method to solve this error, you can share it. I would like to learn another way to resolve this issue.

Thank you.

Steps in Data Preprocessing

Ismiel Abir — Thu, 06 Jun 2024 12:40:01 +0000

Data preprocess

a. missing values
b. smoothing noisy data
c. outliers
d. inconsistency

Data integration

a. data integrates
b. data duplicates
c. data merging
d. data consolidation

Data transformation

a. Scale
b. Normalization
c. Aggregate
d. Generalized

and others steps.

Understanding the Flag Register in x86 Assembly Language

Ismiel Abir — Wed, 17 Apr 2024 23:07:41 +0000

In assembly language, understanding the processor status and flags register is very crucial part. Flag register is the special purpose of register. There are total 9 flags register. In this article, we will describe all this status flag registers. So, lets start….

We can divided the 9 flags into two category, those are

Status Flags

There are 6 flag registers in 8086 microprocessor which become set (1) or reset (0) depending upon condition after either 8-bit or 16-bit operation. These flags are conditional/status flags.

The 6 status flags are:

a. Sign Flag (SF)

b. Zero Flag (ZF)

c. Carry Flag (CF)

d. Auxiliary Carry Flag (AF)

e. Parity Flag (PF)

f. Overflow Flag (OF)

Control Flags

The control flags enable or disable certain operations of the microprocessor.

There are 3 control flags…

a. Directional Flag (DF)

b. Interrupt Flag (IF)

c. Trap Flag (TF)

Sign Flag (SF): If the MSB of a result is 1 than the sign flag will be 1. If the MSB of the result is 0 than the sign flag will be 0.

Zero Flag (ZF): For zero result, it will be 1. For non-zero result it will be 0.

Carry Flag (CF): If there is a carry out from the most significant bit (MSB) on addition, or there is a borrow into the MSB on subtraction, CF will be 1. Otherwise, CF will be 0.

Auxiliary Carry Flag (AF): If there is a carry out from bit 3 on addition, or a borrow into bit 3 on subtraction, AF will be 1. Otherwise, AF will be 0.

Parity Flag (PF): If the low byte of a result has an even number, PF will be 1. Otherwise, PF will be 0.

Overflow Flag (OF): If signed overflow occurred, OF will be 1. Otherwise OF will be 0.

Effect of Instructions on Flags
Different instructions in assembly language may affect the flags differently. For example, arithmetic instructions like ADD and SUB directly affect all flags. INC and DEC flag affects all except CF. On the other hand, MOV and XCHG affects no flags.

References:
Y. Y. Yu and C. Marut, Assembly Language Programming and Organization of the IBM PC, illustrated ed. Mitchell McGraw-Hill, 1992