Recently, I embarked on a learning journey, delving into Data Analysis and Data Science. Using tools such as Python, Numpy, Pandas, Matplotlib, and Seaborn, I decided to solidify my knowledge by undertaking the Data Analysis course at freeCodeCamp, aiming to achieve my first certification.
Simultaneously, I introduced myself to the universe of Elixir Nx. In this new journey, I explored tools like Nx, Explorer, and VegaLite, promising to add a unique dimension to my data analyses.
I will share a detailed tutorial in an upcoming post, providing practical insight into their application.
After completing the certification using Python, the idea of embracing a new challenge, I decided to tackle data analysis problems using the Elixir toolset exclusively.
My first project in this new phase is the Mean-Variance-Standard Deviation Calculator. An exciting opportunity to apply newly acquired concepts and explore the effectiveness of Elixir in handling complex statistical analysis challenges.
By sharing this experience, I hope to document my progress and encourage other enthusiasts to explore multiple languages and tools on their data analysis journey. I am eager to share the results and lessons learned throughout this exciting transition from Python to Elixir. Stay tuned for more updates and practical tutorials!
Setup
Mix.install([
{:nx, "~> 0.6.4"}
])
Challenge
Mean-Variance-Standard Deviation Calculator.
Create a function named calculate() in the MeanVarStd module that uses Elixir Nx to output the mean, variance, standard deviation, max, min, and sum of the rows, columns, and elements in a 3 x 3 matrix.
The input of the function should be a list containing 9 digits. The function should convert the list into a 3 x 3 numerical array, and then return a map containing the mean, variance, standard deviation, max, min, and sum along both axes and for the flattened matrix.
The returned dictionary should follow this format:
%{
mean: [axis1, axis2, flattened],
variance: [axis1, axis2, flattened],
standard_deviation: [axis1, axis2, flattened],
max: [axis1, axis2, flattened],
min: [axis1, axis2, flattened],
sum: [axis1, axis2, flattened],
}
If a list containing less than 9 elements is passed into the function, it should raise a ValueError exception with the message: "List must contain nine numbers." The values in the returned map should be lists and not numerical arrays.
For example, calculate([0,1,2,3,4,5,6,7,8]) should return:
%{
mean: [[3.0, 4.0, 5.0], [1.0, 4.0, 7.0], 4.0],
variance: [
[6.0, 6.0, 6.0],
[0.6666666666666666, 0.6666666666666666, 0.6666666666666666],
6.666666666666667
],
standard_deviation: [
[2.449489742783178, 2.449489742783178, 2.449489742783178],
[0.816496580927726, 0.816496580927726, 0.816496580927726],
2.581988897471611
],
max: [[6, 7, 8], [2, 5, 8], 8],
min: [[0, 1, 2], [0, 3, 6], 0],
sum: [[9, 12, 15], [3, 12, 21], 36]
}
Solution
Create a custom exception:
defmodule ValueError do
defexception message: "bad argument in arithmetic expression"
end
The MeanVarStr module:
defmodule MeanVarStd do
def calculate(list) do
if length(list) == 9 do
tensor =
list
|> Nx.tensor()
|> Nx.reshape({3, 3})
%{
mean: mean(tensor),
variance: variance(tensor),
standard_deviation: standard_deviation(tensor),
max: max(tensor),
min: min(tensor),
sum: sum(tensor)
}
else
raise ValueError
end
end
defp mean(tensor) do
[
tensor
|> Nx.mean(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.mean(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.mean()
|> Nx.to_number()
]
end
defp variance(tensor) do
[
tensor
|> Nx.variance(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.variance(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.variance()
|> Nx.to_number()
]
end
defp standard_deviation(tensor) do
[
tensor
|> Nx.standard_deviation(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.standard_deviation(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.standard_deviation()
|> Nx.to_number()
]
end
defp max(tensor) do
[
tensor
|> Nx.reduce_max(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.reduce_max(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.reduce_max()
|> Nx.to_number()
]
end
defp min(tensor) do
[
tensor
|> Nx.reduce_min(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.reduce_min(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.reduce_min()
|> Nx.to_number()
]
end
defp sum(tensor) do
[
tensor
|> Nx.sum(axes: [0])
|> Nx.to_list(),
tensor
|> Nx.sum(axes: [1])
|> Nx.to_list(),
tensor
|> Nx.sum()
|> Nx.to_number()
]
end
end
Test
ExUnit.start(auto_run: false)
defmodule ExampleTest do
use ExUnit.Case, async: false
# @tag :skip
test "Expected different output when calling 'calculate()' with '[2,6,2,8,4,0,1,5,7]'" do
actual = MeanVarStd.calculate([2, 6, 2, 8, 4, 0, 1, 5, 7])
expected = %{
max: [[8, 6, 7], [6, 8, 7], 8],
mean: [
[3.6666667461395264, 5.0, 3.0],
[3.3333332538604736, 4.0, 4.333333492279053],
3.8888888359069824
],
min: [[1, 4, 0], [2, 0, 1], 0],
standard_deviation: [
[
3.0912060737609863,
0.8164966106414795,
2.943920373916626
],
[
1.8856180906295776,
3.265986442565918,
2.494438409805298
],
2.6434171199798584
],
sum: [[11, 15, 9], [10, 12, 13], 35],
variance: [
[
9.555554389953613,
0.6666666865348816,
8.666666984558105
],
[
3.555555582046509,
10.666666984558105,
6.222222805023193
],
6.987654209136963
]
}
assert actual == expected
end
# @tag :skip
test "Expected different output when calling 'calculate()' with '[9,1,5,3,3,3,2,9,0]'" do
actual = MeanVarStd.calculate([9, 1, 5, 3, 3, 3, 2, 9, 0])
expected = %{
max: [[9, 9, 5], [9, 3, 9], 9],
mean: [
[
4.666666507720947,
4.333333492279053,
2.6666667461395264
],
[5.0, 3.0, 3.6666667461395264],
3.8888888359069824
],
min: [[2, 1, 0], [1, 3, 0], 0],
standard_deviation: [
[
3.0912060737609863,
3.399346351623535,
2.054804801940918
],
[3.265986442565918, 0.0, 3.858612298965454],
3.034777879714966
],
sum: [[14, 13, 8], [15, 9, 11], 35],
variance: [
[
9.55555534362793,
11.555556297302246,
4.222222328186035
],
[10.666666984558105, 0.0, 14.888888359069824],
9.20987606048584
]
}
assert actual == expected
end
# @tag :skip
test "List must contain nine numbers." do
assert_raise ValueError, "bad argument in arithmetic expression", fn ->
MeanVarStd.calculate([2, 6, 2, 8, 4, 0, 1])
end
end
end
Execute Test
PS: Remember to remove the @tag :skip.
ExUnit.run()
...
Finished in 0.00 seconds (0.00s async, 0.00s sync)
3 tests, 0 failures
Randomized with seed 433319
You can check out my livebook solution.
Top comments (2)
A really interesting article. Thanks for sharing!
Thanks, @airled glad you like the post! <3