DEV Community: Nilotpal Choudhury

Adding new column to existing DataFrame in Python pandas

Nilotpal Choudhury — Thu, 26 Aug 2021 15:50:04 +0000

Adding new column to existing DataFrame in Python pandas

Sep 23 '12 Comments: 1 Answers: 29

I have the following indexed DataFrame with named columns and rows not- continuous numbers:

          a         b         c         d
2  0.671399  0.101208 -0.181532  0.241273
3  0.446172 -0.243316  0.051767  1.577318
5  0.614758  0.075793 -0.451460 -0.012493

I would like to add a new column, 'e', to the existing data frame and do…

Open Full Question

Answer: pandas add higher level column multiindex

Nilotpal Choudhury — Thu, 26 Aug 2021 09:09:32 +0000

answer re: pandas add higher level column multiindex

Aug 15 '20

I believe you need in first step convert first level to date values and then reshape by DataFrame.unstack, MultiIndex.reorder_levels and DataFrame.sort_index:

date = pd.Timestamp('today').floor('d')
df.index = pd.MultiIndex.from_product([[date], df.index.get_level_values(1)])
df = df.unstack().reorder_levels([2,0,1], axis=1).sort_index(axis=1)
print (df)
company    Bank X                        Bank Y                        Bank Z  \
product         A        B        C           A        B

…

Open Full Answer

Answer: Replacing blank values (white space) with NaN in pandas

Nilotpal Choudhury — Mon, 23 Aug 2021 15:12:43 +0000

answer re: Replacing blank values (white space) with NaN in pandas

Feb 21 '14

260

I think df.replace() does the job, since pandas 0.13:

df = pd.DataFrame([
    [-0.532681, 'foo', 0],
    [1.490752, 'bar', 1],
    [-1.387326, 'foo', 2],
    [0.814772, 'baz', ' '],     
    [-0.222552, '   ', 4],
    [-1.176781,  'qux', '  '],         
], columns='A B C'.split(), index=pd.date_range('2000-01-01','2000-01-06'))

# replace field that's entirely space (or empty) with NaN
print(df.replace(r'^\s*$',

…

Open Full Answer

Answer: Pandas - How to flatten a hierarchical index in columns

Nilotpal Choudhury — Mon, 23 Aug 2021 14:20:22 +0000

answer re: Pandas - How to flatten a hierarchical index in columns

Apr 19 '19

119

All of the current answers on this thread must have been a bit dated. As of pandas version 0.24.0, the .to_flat_index() does what you need.

From panda's own documentation:

MultiIndex.to_flat_index()

Convert a MultiIndex to an Index of Tuples containing the level values.

A simple example from its documentation:

import

…

Open Full Answer

Answer: sum(-1) in np.array.sum(-1) mean? [duplicate]

Nilotpal Choudhury — Tue, 17 Aug 2021 17:28:04 +0000

answer re: sum(-1) in np.array.sum(-1) mean?

Jun 11 '20

According to numpy doc, It is sum along the axis=-1 which in python is last axis. In python indexing, -1 represents the last index. In your example, it is the sum of all elements along the 3rd axis, which results in a (10,10) shape. Here is a smaller example…

Open Full Answer

Answer: How to import python file located in same subdirectory in a pycharm project

Nilotpal Choudhury — Tue, 27 Jul 2021 05:19:46 +0000

answer re: How to import python file located in same subdirectory in a pycharm project

Mar 11 '16

If you don't have an __init__.py create one and add this line

from util.util import my_function

then you can easily import the module in your scripts the __init__.py tells python that it should treat that folder as a python package, it can also be used to import/load modules too.

in…

Open Full Answer

Answer: How do I update\upgrade pip itself from inside my virtual environment?

Nilotpal Choudhury — Thu, 27 May 2021 08:10:57 +0000

answer re: How do I update\upgrade pip itself from inside my virtual environment?

Mar 1 '16

The more safe method is to run pip though a python module:

python -m pip install -U pip

On windows there seem to be a problem with binaries that try to replace themselves, this method works around that limitation.

Open Full Answer

Answer: How to sort pandas dataframe by custom order on string index

Nilotpal Choudhury — Fri, 05 Mar 2021 20:49:50 +0000

Sep 15 '20

As of Pandas 1.1 DataFrame.sort_values has a key param that takes a callable to control sorting. So you could use an approach like the following:

def sorter(column)
    reorder = [
        "Maurice Baker",
        "Adrian Caldwell",
        "Ratko Varda",
        "Ryan Bowen",
        "Cedric Hunter",
    ]
    # This also works:
    # mapper = {name: order

…

Open Full Answer

Answer: How to upgrade all Python packages with pip

Nilotpal Choudhury — Mon, 22 Feb 2021 03:39:44 +0000

answer re: How to upgrade all Python packages with pip

Apr 30 '11

718

You can use the following Python code. Unlike pip freeze, this will not print warnings and FIXME errors For pip < 10.0.1

import pip
from subprocess import call

packages = [dist.project_name for dist in pip.get_installed_distributions()]
call("pip install --upgrade " + ' '.join(packages), shell=True)

For pip >= 10.0.1

import pkg_resources

…

Open Full Answer

Answer: How to change the X axis range in seaborn in python?

Nilotpal Choudhury — Wed, 11 Nov 2020 10:40:26 +0000

answer re: How to change the X axis range in seaborn in python?

Feb 22 '19

For the most flexible control with these kind of plots, create your own axes object then add the seaborn plots to it. Then you can perform the standard matplotlib changes to features like the x-axis, or use any of the normal controls available through the matplotlib API.

import matplotlib.pyplot as

…

Open Full Answer

Answer: correlation between arrays in python

Nilotpal Choudhury — Tue, 10 Nov 2020 17:42:15 +0000

answer re: correlation between arrays in python

Feb 21 '14

You need numpy.corrcoef:

In [8]:

np.corrcoef(a1,a2)
Out[8]:
array([[ 1.        ,  0.98198051],
       [ 0.98198051,  1.        ]])

Open Full Answer

Answer: How do I change the figure size for a seaborn plot?

Nilotpal Choudhury — Tue, 10 Nov 2020 08:39:49 +0000

answer re: How do I change the figure size for a seaborn plot?

Dec 23 '17

271

You can also set figure size by passing dictionary to rc parameter with key 'figure.figsize' in seaborn set method:

import seaborn as sns

sns.set(rc={'figure.figsize':(11.7,8.27)})

Other alternative may be to use figure.figsize of rcParams to set figure size as below:

from matplotlib import rcParams

# figure size in inches
rcParams['figure.figsize'] =

…

Open Full Answer