Skip to content

too low precision for double data points #395

Open
@DNKpp

Description

@DNKpp

Bug category

  • bug - compilation error
  • bug - compilation warning
  • bug - runtime error
  • bug - runtime warning
  • bug - logic error

Describe the bug

The precision used for converting the double data values from binary to textual representation seems to unreasonable low. That's at least the case in the line::data_string function, where it is constantly set to 10 (see line.cpp#L166).
This leads to data points not showing up in plots with logarithmic axes scaling.

Locally I bumped this straight to 16 (std::numeric_limits<double>::digits10yields 15),, which works as intended (at least for me).

Steps to Reproduce
Well, create a loglog plot with very small double data values.

Output

imprecise output

image
That's part of the cmd dump, sent to gnuplot
image

fixed output

image
That's part of the cmd dump, sent to gnuplot
image

Platform

  • cross-platform issue - linux
  • cross-platform issue - windows
  • cross-platform issue - macos

Environment Details:

  • OS: Windows
  • OS Version: 11
  • Compiler: msvc
  • Compiler version: 17.9.0

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions