Skip to content

Add support for libmagic. #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
5242f32
Added Scheme level binding for mdb_del.
0-8-15 Jul 29, 2018
a66dca9
Merge branch 'master' of github.com:part-cw/lambdanative
0-8-15 Aug 4, 2018
3271306
Merge branch 'master' of github.com:0-8-15/lambdanative
0-8-15 May 7, 2019
dde5b32
Update gambit to version 4.9.2
May 15, 2019
133e66b
Merge branch 'master' of ssh://github.com/part-cw/lambdanative
May 15, 2019
28170f6
update experimental libgamit
0-8-15 May 19, 2019
caa20a2
Merge branch 'master' of ssh://github.com/0-8-15/lambdanative
May 19, 2019
42754eb
Merge branch 'master' of ssh://github.com/part-cw/lambdanative
0-8-15 May 29, 2019
f802f9f
Do not treat syntax-case module special anymore.
0-8-15 May 21, 2019
cc258f0
Include unistd.h to silence warning wrt readlink on Linux.
0-8-15 Jun 4, 2019
5d8868a
Merge branch 'master' of ssh://github.com/part-cw/lambdanative into m…
0-8-15 Jul 11, 2019
500f290
Merge branch 'master' of ssh://github.com/part-cw/lambdanative into m…
0-8-15 Jul 11, 2019
b63b2a5
Add support for libmagic.
0-8-15 Jul 13, 2019
1aa159f
Merge branch 'add-libmagic' into master-updates
0-8-15 Jul 13, 2019
ea4b93b
Add support for libmagic.
0-8-15 Jul 13, 2019
4fa3909
Update openssl to 1.1.c
0-8-15 Jul 13, 2019
9131bda
Merge branch 'update-openssl' into master-updates
0-8-15 Jul 13, 2019
220cf6a
Minor improvements to build process.
0-8-15 Jul 13, 2019
093af12
exclude devcryptoeng for macosx
0-8-15 Jul 16, 2019
8c4f3c8
merged upstream changes
0-8-15 Jul 16, 2019
cde9100
Merge branch 'master-updates' of ssh://github.com/0-8-15/lambdanative…
0-8-15 Jul 16, 2019
fa61df7
Merge remote-tracking branch 'upstream/master' into master-updates
0-8-15 Jul 18, 2019
a933c5d
allow libiconv cross-compile for win32 on linux using mingw32
0-8-15 Aug 6, 2019
ae22ff0
Merge branch 'master' of ssh://github.com/part-cw/lambdanative into m…
0-8-15 Aug 8, 2019
6968521
Compare string in sh with '=' not '=='
0-8-15 Aug 15, 2019
e883722
Merge branch 'master' of https://github.com/part-cw/lambdanative into…
0-8-15 Aug 15, 2019
7871d11
Avoid building engines not available on wingw
0-8-15 Aug 15, 2019
554df3c
Overwrite make-thread with make-safe-thread
0-8-15 Aug 15, 2019
8845ccc
At least produce a debug log entry when quirc failes to decode.
0-8-15 Aug 15, 2019
86dcebc
merged upstream
0-8-15 Aug 16, 2019
9d3e141
Merge branch 'master' of ssh://github.com/part-cw/lambdanative into m…
0-8-15 Aug 19, 2019
79398d7
Handle compilation and permissions for ANDROIDAPI>22.
0-8-15 Aug 29, 2019
6a2bcdc
Do not break syntax highlighting by `ifdef` alike trick.
0-8-15 Aug 29, 2019
78a1cd1
Merge branch 'master' of ssh://github.com/part-cw/lambdanative into m…
0-8-15 Sep 2, 2019
8dd4432
Name mutex's to ease debugging.
0-8-15 Sep 4, 2019
e4a9372
Err on usability wrt. clipboard content.
0-8-15 Sep 5, 2019
b936245
Enable use of cond-expand depending on SYS_PLATFORM.
0-8-15 Sep 6, 2019
2e8d26b
LN_STORE: Name mutex with the symbol they are bound to. (#262)
0-8-15 Sep 7, 2019
7e63385
Disable vpx on win32 since it did not build with it.
0-8-15 Sep 22, 2019
c4dddb9
Changes to windows build.
0-8-15 Sep 22, 2019
3366268
add patch for win32
0-8-15 Sep 22, 2019
9189dfa
was the EMBED file forgotton?
0-8-15 Sep 22, 2019
e460673
Add support for libmagic.
0-8-15 Sep 22, 2019
bfcdab2
libpcre: enable compile with clang
0-8-15 Sep 22, 2019
f8f278c
Use embedded magic.mgc by default
0-8-15 Sep 22, 2019
dccce2d
Load embedded magic.mgc if called with #t
0-8-15 Sep 26, 2019
b1230e2
Merge branch 'libmagic' into add-libmagic
0-8-15 Sep 26, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 2 additions & 18 deletions languages/scm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,7 @@ compile_payload_scm()
scm_coresrcs=
scm_auxsrcs=
for m in $modules; do
if [ $m = "syntax-case" ]; then
scm_modsrc="$SYS_HOSTPREFIX/lib/syntax-case.scm"
else
scm_modsrc=`locatefile modules/$m/$m.scm silent`
fi
scm_modsrc=`locatefile modules/$m/$m.scm silent`
if [ `string_contains "$scm_coremodules" " $m "` = yes ]; then
scm_coresrcs="$scm_coresrcs $scm_modsrc"
else
Expand All @@ -56,6 +52,7 @@ compile_payload_scm()
else
scm_opts="(declare (block)(not safe)(standard-bindings)(extended-bindings))"
fi
scm_opts="${scm_opts}(define-cond-expand-feature $SYS_PLATFORM)"
# support global macro definitions
if [ -f "${SYS_HOSTPREFIX}/lib/global-macros.scm" ]; then
scm_opts="${scm_opts}(include \\\"~~lib/global-macros.scm\\\")"
Expand All @@ -65,19 +62,6 @@ compile_payload_scm()
payload_cdefs="$payload_cdefs -DSTANDALONE"
fi
#--------
# syntax-case special-case
if [ `string_contains "$modules" "syntax-case"` = yes ]; then
if [ ! -f ${SYS_HOSTPREFIX}/lib/gambcext.tmp ]; then
echo " => compiling syntax-case dynamic library.."
veval "$SYS_GSC -dynamic -o ${SYS_HOSTPREFIX}/lib/gambcext.o1 ${SYS_HOSTPREFIX}/lib/syntax-case.scm"
mv ${SYS_HOSTPREFIX}/lib/gambcext.o1 ${SYS_HOSTPREFIX}/lib/gambcext.tmp
fi
assertfile ${SYS_HOSTPREFIX}/lib/gambcext.tmp
cp ${SYS_HOSTPREFIX}/lib/gambcext.tmp ${SYS_HOSTPREFIX}/lib/gambcext.o1
else
rmifexists ${SYS_HOSTPREFIX}/lib/gambcext.o1
fi
#--------
# compile scheme source files
mkdir -p "$SYS_PREFIX/build"
scm_csrcs=
Expand Down
Empty file modified libraries/libgambit/make.sh
100755 → 100644
Empty file.
2 changes: 1 addition & 1 deletion libraries/libgd/make.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ EXTRACONF=
if [ ! $SYS_PLATFORM = $SYS_HOSTPLATFORM ]; then
EXTRACONF=--host=$SYS_ARCH
fi
if [ $SYS_PLATFORM = android ]; then
if [ $SYS_PLATFORM = android -o $SYS_PLATFORM = win32 ]; then
EXTRACONF="$EXTRACONF --without-vpx"
fi

Expand Down
1 change: 1 addition & 0 deletions libraries/libmagic/LIB_DEPENDS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
libpcre
76 changes: 76 additions & 0 deletions libraries/libmagic/make.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
PKGURL=https://github.com/threatstack/libmagic/archive/5.18.tar.gz
PKGHASH=ce734a1cc24bddbfd8b21f4ffdfe721ab74eeed9

package_download $PKGURL $PKGHASH
package_patch

EXTRACONF=

if [ "$SYS_PLATFORM" != "$SYS_HOSTPLATFORM" ]; then
EXTRACONF="$EXTRACONF --host=$SYS_ARCH"
fi

# pretend libgnurx exists by masking libpcre

test -L $SYS_PREFIX/include/regex.h || ln -s pcreposix.h $SYS_PREFIX/include/regex.h

# Mabye instead of fooling the package, it should be patched
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure about this either. Why not include that library if you depend on it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two reasons:

  1. At least I need pcre anyway and I see limited reasons to have two libraries implementing the same functionality.

  2. I'd guess there might be license compatibility issues when linking against a GPLed library and libgnurx is AFAIK.

rmifexists $SYS_PREFIX/lib/libgnurx.a
rmifexists $SYS_PREFIX/lib/libgnurx
mkdir $SYS_PREFIX/lib/libgnurx
( cd $SYS_PREFIX/lib/libgnurx
mkdir a
cd a
$SYS_AR -x ../../libpcre.a
cd ..
mkdir b
cd b
$SYS_AR -x ../../libpcreposix.a
cd ..
$SYS_AR rc ../libgnurx.a a/*.o b/*.o
)

rm -r $SYS_PREFIX/lib/libgnurx

if [ `file --version|head -1` != file-5.18 ]; then
if [ ! -f $SYS_HOSTPREFIX/bin/file ]; then
./configure --enable-static --disable-shared 'CFLAGS=-fPIC -O3 -DPCRE_STATIC'
package_make
cp src/file $SYS_HOSTPREFIX/bin || exit 1
package_cleanup
package_download $PKGURL $PKGHASH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't it already be downloaded by this time (as in line 4)?

Copy link
Contributor Author

@0-8-15 0-8-15 Jul 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it should. Reading package.sh again I now see that this package_download might be better replaced by a package_unpack. I just was not aware of the latter being available as a separate call.

package_patch
fi
FILE_COMPILE=$SYS_HOSTPREFIX/bin/file
fi

# configure

if [ "$SYS_PLATFORM" != "$SYS_HOSTPLATFORM" -a "X$FILE_COMPILE" != "X" ]; then
echo patching magic/Makefile.in to use $FILE_COMPILE
sed -i -es%'@IS_CROSS_COMPILE_TRUE@FILE_COMPILE = file${EXEEXT}'%"@IS_CROSS_COMPILE_TRUE@FILE_COMPILE = ${FILE_COMPILE}"% magic/Makefile.in
fi

package_configure --enable-static --disable-shared $EXTRACONF "'CFLAGS=-fPIC -O3 -DPCRE_STATIC -DNOSHLWAPI'"


# build

package_make

# install

cp src/.libs/libmagic.a $SYS_PREFIX/lib
cp src/magic.h $SYS_PREFIX/include
test -f src/file.exe && cp src/file.exe $SYS_PREFIX/bin
test -d $SYS_PREFIX/etc || mkdir $SYS_PREFIX/etc
cp magic/magic.mgc $SYS_PREFIX/etc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need this file embedded couldn't you compile it once and then include it in the module with an EMBED file? - Also does it need to be in the /etc folder?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. I don't really know where to put it. As you said: embedding might not be available from within the library directory. Hence we'll have to install it somewhere to be available for embedding later.


# Cleanup the fooling
rm -f $SYS_PREFIX/lib/libgnurx.a $SYS_PREFIX/include/regex.h

package_cleanup

unset EXTRACONF

#eof
20 changes: 20 additions & 0 deletions libraries/libmagic/no_use_PathRemoveFileSpecA.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
--- src/magic.original.c 2014-08-28 17:53:19.000000000 +0200
+++ src/magic.c 2019-08-07 17:56:04.790561769 +0200
@@ -175,7 +175,16 @@
LPTSTR dllpath = malloc(sizeof(*dllpath) * (MAX_PATH + 1));
dllpath[MAX_PATH] = 0; /* just in case long path gets truncated and not null terminated */
if (GetModuleFileNameA(NULL, dllpath, MAX_PATH)){
- PathRemoveFileSpecA(dllpath);
+ // PathRemoveFileSpecA(dllpath);
+ {
+ // PathRemoveFileSpecA is not avail on mingw
+ int i = strlen(dllpath);
+ while(i>0) {
+ if(dllpath[i] == '\\') break;
+ dllpath[i--] = '\0';
+ }
+ dllpath[i] = '\0';
+ }
if (strlen(dllpath) > 3 &&
stricmp(&dllpath[strlen(dllpath) - 3], "bin") == 0) {
if (asprintf(&tmppath,
36 changes: 36 additions & 0 deletions libraries/libpcre/make.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
PKGURL=https://ftp.pcre.org/pub/pcre/pcre-8.43.tar.gz
PKGHASH=8f36ed69d3e938972fc511c19bfaa0ff27ff1d71

package_download $PKGURL $PKGHASH

package_patch

EXTRACONF="--enable-utf --enable-jit "

if [ "$SYS_PLATFORM" != "$SYS_HOSTPLATFORM" ]; then
EXTRACONF="$EXTRACONF --host=$SYS_ARCH"
fi

# configure

lncc=`echo $SYS_CC| cut -d ' ' -f 1`
case $lncc in
*clang) deflncxx="CXX=${lncc}++" ;;
esac
package_configure $EXTRACONF "CC=$lncc $deflncxx"

# build

NOQUIET=yes package_make

# install

cp .libs/libpcre.a .libs/libpcreposix.a $SYS_PREFIX/lib || exit 1
cp pcre.h pcreposix.h pcrecpparg.h pcre_scanner.h pcre_stringpiece.h $SYS_PREFIX/include || exit 1

package_cleanup

unsset deflncxx
unset EXTRACONF

#eof
Loading