DEV Community

Caleb Zhao
Caleb Zhao

Posted on

Incorrect calculations: sec(x) near x=k*π+π/2

In this blog post, we explore incorrect calculations of the secant function sec(x) in MATLAB near numbers around kπ+π2. k\,\pi+\frac{\pi}{2}.

Example 1. Given that x1=0.1099557429e23π+π2 x_1=0.1099557429\textup{e}2 \approx 3\,\pi+\frac{\pi}{2} and
x2=0.8011061266654e225π+π2,x_2=0.8011061266654\textup{e}2 \approx 25\,\pi+\frac{\pi}{2}\,, calculate sec(x1) \sec(x_1) and sec(x2) \sec(x_2) in MATLAB.

Using MATLAB, we computed the values of sec(x1) \sec(x_1) and sec(x2). \sec(x_2). The result is shown in the following screenshot.

MATLAB outputFrom the above screenshot, it can be seen that the outputs from MATLAB are
4.105555334642597e+084.10555\red{5334642597}\textup{e}+08 and 3.580150224915013e+123.\red{580150224915013}\textup{e}+12 , respectively.

However, the correct values with 16 significant digits are 0.4105556037464873e9 and 0.3670813182326778e13, respectively (as provided by ISRealsoft).

Thus, the output of MATLAB contains only 6 and 1 correct digits, with error rates of (16-6)/16 = 62.5% and (16-1)/16 = 93.75% respectively for the significant figures.

previous \,\quad\quad\quad\quad\hspace{7.2cm} next

Image of Datadog

The Essential Toolkit for Front-end Developers

Take a user-centric approach to front-end monitoring that evolves alongside increasingly complex frameworks and single-page applications.

Get The Kit

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up