Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: meson build #120

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
63e93f5
WIP: meson build
vtorri May 2, 2024
fc52493
improve meson code
vtorri May 2, 2024
72f696e
remove '== true' or '== false', use 'in' instead of 'contains()'
vtorri May 3, 2024
8f0731e
remove now useless 'res', formatting
vtorri May 3, 2024
7056bbc
remove uselesss match_finders array, + fix
vtorri May 3, 2024
d823821
remove uselesss checks array
vtorri May 3, 2024
3f29010
remove outer if
vtorri May 3, 2024
070a65d
change project name to 'xz-utils' to avoid space in the dist tarball …
vtorri May 4, 2024
132cde1
use get_option() instead of array
vtorri May 4, 2024
fc9655e
fix copy-paste typo
vtorri May 4, 2024
0ffc20d
fix posix_fadvise() detection
vtorri May 9, 2024
aca1aa2
change tarball name to 'xz' as recommended in PACKAGERS file
vtorri May 9, 2024
ae73fa4
add xzdec and lzmadec + a couple of improvements
vtorri May 15, 2024
0f2859d
add xz tool + some fixes
vtorri May 16, 2024
96e2d96
remove uint32_t check
vtorri May 16, 2024
f51a11f
move C99 header macros to top level meson.build, unconditionally set …
vtorri May 16, 2024
be2a23a
indent #
vtorri May 16, 2024
8a7b9b1
fix description of 'unsafe-type-punning' option
vtorri May 16, 2024
a83a638
use cc.has_function_attribute instead of compile C code
vtorri May 16, 2024
859f821
use .set() method instead of .set10(), except for visibility macro
vtorri May 18, 2024
fb6b458
change description of macro, define _GNU_SOURCE and __EXTENSIONS__
vtorri May 18, 2024
24ad34e
remove check of wchar.h for wcwidth()
vtorri May 18, 2024
efcb02e
remove double # at the top of meson.build files
vtorri May 18, 2024
d45e486
add lzmainfo and scripts, plus some fixes
vtorri May 22, 2024
7105a71
add C tests
vtorri May 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
343 changes: 343 additions & 0 deletions meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,343 @@
## SPDX-License-Identifier: 0BSD
## Author: Vincent Torri

project(
'xz-utils',
vtorri marked this conversation as resolved.
Show resolved Hide resolved
'c',
version: '5.6.99',
license: '0BSD',
license_files: [
'COPYING',
'COPYING.0BSD',
'COPYING.GPLv2',
'COPYING.GPLv3',
'COPYING.LGPLv2.1'
],
default_options: [
'b_ndebug=if-release',
'c_std=c11',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Meson 1.3.0 it's possible to have a list. For example, c_std=gnu99,c11,c99 would prefer gnu99 but then try c11 before c99. I don't know if MSVC counts as C99 as it officially supports only C11, I think, perhaps due to C99 having a few mandatory features that are optional in C11.

GNU extensions are used when they are available, thus gnu99 can make sense. gnu99 vs. c99 also affect the predefined macros. On my system with GCC 13.2.1, -std=c99 defines __STRICT_ANSI__ while -std=gnu99 doesn't. It can then affect the behavior of system headers.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that some project leaders do not like to use the latest version of meson. If there is no problem for you to use meson >= 1.3, then ok.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know right now which Meson (or muon) version is OK to require. I just get an impression that a single value doesn't work everywhere (if MSVC needs C11 mode). I haven't checked in detail how the c_std option in Meson exactly works.

Copy link

@eli-schwartz eli-schwartz May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/mesonbuild/meson/blob/master/mesonbuild/compilers/c.py#L453-L485

For MSVC, meson ignores c99 and passes /std:XXX when the mode is c11 or c17. The difference is really about whether you want to opt into GNU extensions (which MSVC obviously doesn't support). Doing that might require bumping the minimum meson requirement, yes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vaguely remember that, a very long time ago, using gnu99 was better in practice when the intention was to use non-standard extensions (language or library) when they are available.

If gnu99 doesn't put MSVC into C11/C17 mode then MSVC needs some other c_std option. MSVC is a special case in so many ways that handling it specially here could be OK too, assuming it's easy to do.

Using gnu11 with GCC and Clang and strict C11 with compilers that don't support GNU extensions could be OK too. But if compiler doesn't support C11 then it should fallback to C99 if that is supported.

The list method that requires Meson >= 1.3.0 would make all this trivially easy. muon says it's currently at meson compat version 1.1.

So, oh well, maybe try gnu11 for now if that helps getting MSVC to C11 mode. This is a small detail in the big picture and can be worried more a little later.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I did for muon with cl.exe compiler: I just pass the required standard. So, no problem with c11. (there will be a message from the compiler if the standard is not correct)
I maybe should pass the standard if it is only c11 or c17 like meson. I'll ask the author

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Meson docs give an impression that c_std=gnu11 would fallback to C11 with MSVC but this might become deprecated and eventually an error. So one would need to list the possible standards.

I think requiring new enough Meson and using c_std=gnu99,c11,c99 or c_std=gnu11,gnu99,c11,c99 is fine for now. It might not be muon compatible right now but I would expect this kind of feature to get implemented at some point in muon too. Very probably the first versions of Meson support won't be as portable as the Autotools build is and it doesn't need to be. Progress happens one step at a time.

'warning_level=3',
],
meson_version: '>= 1.1.0',
)

v_array = meson.project_version().split('.')
v_maj = v_array[0]

# config.h

config_dir = [include_directories('.')]

config_h = configuration_data()
config_h.set_quoted('PACKAGE', 'xz')
config_h.set_quoted('PACKAGE_BUGREPORT', '[email protected]')
config_h.set_quoted('PACKAGE_NAME', meson.project_name())
config_h.set_quoted('PACKAGE_URL', 'https://tukaani.org/xz/')

# host

config_h.set10('WORDS_BIGENDIAN', host_machine.endian() == 'big',
description: 'Define to 1 if the processor stores words with the most significant byte first.'
)
vtorri marked this conversation as resolved.
Show resolved Hide resolved

host_os = host_machine.system()

cygwin = 'cygwin'
windows = 'windows'
linux = 'linux'
sunos = 'sunos'
asm_os = [ 'linux', 'dragonfly', 'freebsd', 'netbsd', 'openbsd', 'windows', 'cygwin' ]
sys_cygwin = cygwin.contains(host_os)
sys_windows = windows.contains(host_os)

COND_W32 = sys_cygwin or sys_windows

# binaries
cc = meson.get_compiler('c')

# try mimic AC_USE_SYSTEM_EXTENSIONS
if host_machine.system() == 'linux'
add_project_arguments('-D_GNU_SOURCE', language: 'c')
vtorri marked this conversation as resolved.
Show resolved Hide resolved
elif host_machine.system() == 'sunos'
add_project_arguments('-D__EXTENSIONS__', language: 'c')
add_project_arguments('-D_POSIX_PTHREAD_SEMANTICS', language: 'c')
else
add_project_arguments('-D_TANDEM_SOURCE', language: 'c')
add_project_arguments('-D_ALL_SOURCE', language: 'c')
endif

lzma_c_args = []
cflags_try = []

if cc.get_argument_syntax() == 'gcc'
cflags_try += [
'-Wvla',
'-Wformat=2',
'-Winit-self',
'-Wmissing-include-dirs',
'-Wshift-overflow=2',
'-Wstrict-overflow=3',
'-Walloc-zero',
'-Wduplicated-cond',
'-Wfloat-equal',
'-Wundef',
'-Wshadow',
'-Wpointer-arith',
'-Wbad-function-cast',
'-Wwrite-strings',
'-Wdate-time',
'-Wsign-conversion',
'-Wfloat-conversion',
'-Wlogical-op',
'-Waggregate-return',
'-Wstrict-prototypes',
'-Wold-style-definition',
'-Wmissing-prototypes',
'-Wmissing-declarations',
'-Wredundant-decls',
#
vtorri marked this conversation as resolved.
Show resolved Hide resolved
'-Wc99-compat',
'-Wc11-extensions',
'-Wc2x-compat',
'-Wc2x-extensions',
'-Wpre-c2x-compat',
'-Warray-bounds-pointer-arithmetic',
'-Wassign-enum',
'-Wconditional-uninitialized',
'-Wdocumentation',
'-Wduplicate-enum',
'-Wempty-translation-unit',
'-Wflexible-array-extensions',
'-Wmissing-variable-declarations',
'-Wnewline-eof',
'-Wshift-sign-overflow',
'-Wstring-conversion',
]

lzma_c_args += cc.get_supported_arguments(cflags_try)
endif

config_h.set10('HAVE_VISIBILITY', cc.has_function_attribute('visibility'),
description: 'Define to 1 if the compiler supports simple visibility declarations.'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eli-schwartz: Elsewhere there is gnu_symbol_visibility: 'hidden'. Is it possible to use the result of that here somehow? I remember that on some platforms the attributes are accepted but produce compiler warnings as they aren't truly supported. I don't know if that is a case on any modern platform though (Cygwin was like that many years ago but I haven't tested in years).

Gnulib's visibility.m4 checks that both attribute and compiler option are supported using -Werror to enable visibility support. Meson's documentation doesn't tell if compiler.has_function_attribute() accepts the attribute if using it produces warnings but compiling still succeeds.

It's also a bit unfortunate if visibility support requires an option in both library and a separate check for the attribute support. Perhaps the gnu_symbol_visibility option could insert preprocessor macros like MESON_GNU_SYMBOL_VISIBILITY_HIDDEN __attribute__((__visibility("hidden"))) which the C/C++ code could use. I'm not sure what is the best way as perhaps this idea has downsides. And modern compilers have __has_attribute anyway.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meson internally handles -Werror for exactly that reason:
.https://github.com/mesonbuild/meson/blob/7d28ff29396f9d7043204de8ddc52226b9903811/mesonbuild/compilers/mixins/clike.py#L1309-L1324

Most projects I have seen, actually check for __GNUC__ instead of using visibility.m4 even with autotools. This is kind of necessary anyway since you need preprocessor logic to figure out whether Microsoft dllexport/dllimport is viable, and visibility.m4 is ultimately not all that helpful overall.

Certain versions of... SunPro, if I recall correctly, are the only commonly known compiler that handles visibility but isn't either __GNUC__ or MSC_VER, and they use a third syntax instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meson internally handles -Werror for exactly that reason:

Great! Documenting it could be nice to give more confidence. Or perhaps it's that coming from other build systems one is constantly suspecting that corner cases aren't handled well enough, sorry.

Most projects I have seen, actually check for __GNUC__ instead of using visibility.m4 even with autotools.

liblzma checks in this order in the C code:

  1. #if defined(_WIN32) || defined(__CYGWIN__) then use __declspec(dllexport).

  2. #if HAVE_VISIBILITY to use GNU C attributes. Checking for __GNUC__ wouldn't work as it can be defined on platforms that don't support the visibility attributes; one has to know if the attribute is actually supported.

  3. Otherwise assume no visibility support.

Certain versions of... SunPro, if I recall correctly, are the only commonly known compiler that handles visibility but isn't either __GNUC__ or MSC_VER, and they use a third syntax instead?

My note in b0d1422 says that SunPro (or whatever its name is this year) supports the GNU C attribute since 2007. At least modern releases support the -fvisibility option too. I didn't research when the command line option was added (it would make sense to add it when the attribute was but I don't know if that happened). The original option is -xldscope but I guess there's no point to add support for it when -fvisibility works too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This

__attribute__((dllexport)) int foo(int x) { return x; }

results in a warning with GCC 14.1.1 on x86-64 GNU/Linux:

warning: ‘dllexport’ attribute directive ignored [-Wattributes]
    1 | __attribute__((dllexport)) int foo(int x) { return x; }
      | ^~~~~~~~~~~~~

But cc.has_function_attribute('dllexport') is still true. So -Werror isn't used or Meson shortcircuits the attribute check somehow. Thus I fear it might do the same with the visibility attribute as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, I'm wrong. I just fell into the exact trap of #undef vs. #define being too easy to mix in config.h. So the previous message is noise. Sorry.


ctor_fct_src = '''
__attribute__((__constructor__))
static void my_constructor_func(void) { return; }
int main() { return 0; }
'''
if cc.compiles(ctor_fct_src, name : 'constructor attribute for functions')
config_h.set10('HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR', true,
description: 'Define to 1 if __attribute__((__constructor__)) is supported for functions.'
)
endif
vtorri marked this conversation as resolved.
Show resolved Hide resolved

cc.has_type('uint32_t', prefix: '#include <stdint.h>')
vtorri marked this conversation as resolved.
Show resolved Hide resolved

time_fcts = [
[ 'futimens', [ '#include <sys/stat.h>' ] ],
[ 'futimes', [ '#include <sys/time.h>' ] ],
[ 'futimesat', [ '#include <fcntl.h>', '#include <sys/time.h>' ] ],
[ 'utimes', [ '#include <sys/time.h>' ] ],
[ '_futime', [ '#include <sys/utime.h>' ] ],
[ 'utime', [ '#include <sys/types.h>', '#include <utime.h>' ] ],
]
foreach f : time_fcts
if cc.has_function(f[0], prefix : f[1])
config_h.set10('HAVE_' + f[0].to_upper(), true,
description: 'Define to 1 if the ' + f[0] + '() function is available.'
)
break
endif
endforeach

if cc.has_function('posix_advice', prefix : '#include <fcntl.h>')
config_h.set10('HAVE_POSIX_ADVICE', true,
description: 'Define to 1 if the posix_advice() function is available.'
)
vtorri marked this conversation as resolved.
Show resolved Hide resolved
endif

### Filters

## encoders
encoder_msg = ''
decoder_msg = ''

simple_filters = [
'x86',
'powerpc',
'ia64',
'arm',
'armthumb',
'arm64',
'sparc',
'riscv',
]

# FIXME foreach i in array; set_variable(f'have_@i@', true) endforeach

foreach f : get_option('encoders')
config_h.set10('HAVE_ENCODER_' + f.underscorify().to_upper(), true,
description: 'Define to 1 if ' + f + ' encoder is enabled.'
)
encoder_msg += f + ' '
endforeach

foreach f : get_option('decoders')
config_h.set10('HAVE_DECODER_' + f.underscorify().to_upper(), true,
description: 'Define to 1 if ' + f + ' decoder is enabled.'
)
decoder_msg += f + ' '
endforeach

have_encoder_simple_filters = false
have_decoder_simple_filters = false
foreach f : simple_filters
if f in get_option('encoders')
have_encoder_simple_filters = true
endif
if f in get_option('decoders')
have_decoder_simple_filters = true
endif
endforeach

if ('lzma2' in get_option('encoders') and not ('lzma1' in get_option('encoders'))) or('lzma2' in get_option('decoders') and not ('lzma1' in get_option('decoders')))
error('LZMA2 requires that LZMA1 is also enabled')
endif

if get_option('encoders').length() > 0
config_h.set10('HAVE_ENCODERS', true,
description: 'Define to 1 if at least one encoder has been enabled.'
)
endif

if get_option('decoders').length() > 0
config_h.set10('HAVE_DECODERS', true,
description: 'Define to 1 if at least one decoder has been enabled.'
)
endif

### Match finders

match_finder_msg = ''

if 'lzma1' in get_option('encoders') or 'lzma2' in get_option('encoders')
if get_option('match-finders').length() == 0
error('at least one match finder is required for an LZ-based encoder')
endif

foreach m : get_option('match-finders')
config_h.set10('HAVE_MF_' + m.underscorify().to_upper(), true,
description: 'Define to 1 to enable ' + m + ' match finder.'
)
match_finder_msg += m + ' '
endforeach
endif

### Integrity checks

check_msg = ''

if not ('crc32' in get_option('checks'))
error('For now, the CRC32 check must always be enabled')
endif

foreach c : get_option('checks')
config_h.set10('HAVE_CHECK_' + c.underscorify().to_upper(), true,
description: 'Define to 1 if ' + c + ' integrity check is enabled.'
)
check_msg += c + ' '
endforeach

### microLZMA

if get_option('microlzma') == true
vtorri marked this conversation as resolved.
Show resolved Hide resolved
endif

### .lz (lzip) format support

have_lzip = false
if not ('lzma1' in get_option('encoders'))
lzip_msg = 'no (LZMA1 disabled)'
elif get_option('lzip-decoder')
lzip_msg = 'yes'
have_lzip = true
config_h.set10('HAVE_LZIP_DECODER', true,
description: 'Define to 1 if .lz (lzip) decompression support is enabled.'
)
else
lzip_msg = 'no'
endif

### assembler optimizations

have_assembler_x86 = false
if get_option('assembler')
if host_os in asm_os and host_machine.cpu_family() == 'x86'
have_assembler_x86 = true
endif
endif

### size optimization

if get_option('small')
config_h.set10('HAVE_SMALL', true,
description: 'Define to 1 if optimizing for size.'
)
endif

### threading

have_threads = false

if get_option('threads') == true
if sys_windows == true
if host_machine.cpu_family() == 'x86'
config_h.set10('MYTHREAD_WIN95', true,
description: 'Define to 1 when using Windows 95 (and thus XP) compatible threads. This avoids use of features that were added in Windows Vista.'
)
else
config_h.set10('MYTHREAD_VISTA', true,
description: 'Define to 1 when using Windows Vista compatible threads. This uses features that are not available on Windows XP.'
)
endif
have_threads = true
else
pthread_dep = dependency('threads', required : false)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect usage. dependency('threads') will always "find" a thread dependency.

dependency('threads') isn't designed to check for pthreads. It just allows you to use whatever threading API is available with the selected toolchain, so you should also use it on the if sys_windows branch.

pthreads have to be detected separately, like with cc.check_header('pthread.h').

CC: @eli-schwartz

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At glance this sounds odd. XZ Utils can use pthreads and Windows threads. C11 threads are available on some platforms too but XZ Utils doesn't support those for now (they don't seem to add any practical value for now).

MinGW-w64 supports pthreads and Windows threads, the latter being preferred. Very recent MSVC supports C11 threads in addition to Windows threads.

When using MinGW-w64 and Windows threads, one specifically wants to ensure that the pthreads flags are not used. Thus the Autotools and CMake builds don't add any threading related options to the build when using Windows threads.

Autotools build assumes that pthreads is available on non-Windows, thus the build fails, for example, on MINIX 3 where pthreads isn't available. One has to manually use --disable-threads. Perhaps I should fix it to show an error at configure time with a message that suggest --disable-threads. This would match what CMakeLists.txt already does. I don't want to it to silently disable threading support.

if pthread_dep.found()
config_h.set10('MYTHREAD_POSIX', true,
description: 'Define to 1 when using POSIX threads (pthreads).'
)
have_threads = true
endif
endif
endif


### subdirectories

subdir('src')

configure_file(output: 'config.h', configuration: config_h)

# summary

summary(
{
'OS': host_os,
'endianness': host_machine.endian(),
'assembler': get_option('assembler').to_string('yes', 'no'),
'encoders': encoder_msg,
'decoders': decoder_msg,
'match finders': match_finder_msg,
'checks': check_msg,
'external sha256': get_option('external-sha256').to_string('yes', 'no'),
'microLZMA': get_option('microlzma').to_string('yes', 'no'),
'lzip decompression': lzip_msg,
'small': get_option('small').to_string('yes', 'no'),
'threads': have_threads.to_string('yes', 'no'),
'fast unaligned access': fast_unaligned_access.to_string('yes', 'no'),
'unsafe type punning': get_option('unsafe-type-punning').to_string('yes', 'no')
},
section: 'Configuration Options Summary:',
)
Loading