diff options
| author | Xin LI <delphij@FreeBSD.org> | 2019-11-10 03:44:32 +0000 |
|---|---|---|
| committer | Xin LI <delphij@FreeBSD.org> | 2019-11-10 03:44:32 +0000 |
| commit | 4ef4bf0189e5e736a37f9ead9beaaa8e1cd1f9fb (patch) | |
| tree | 0b82394b9b244b8f3a588afe3b96177c08ff98eb | |
| parent | ad1ba6e1e032f79d03b9a6e2c714de84d1911b3a (diff) | |
| download | src-4ef4bf0189e5e736a37f9ead9beaaa8e1cd1f9fb.tar.gz src-4ef4bf0189e5e736a37f9ead9beaaa8e1cd1f9fb.zip | |
Vendor import of file 5.37vendor/file/5.37
Notes
Notes:
svn path=/vendor/file/dist/; revision=354582
svn path=/vendor/file/5.37/; revision=354583; tag=vendor/file/5.37
233 files changed, 5340 insertions, 1987 deletions
@@ -1 +1 @@ -See COPYING.
\ No newline at end of file +See COPYING. @@ -1,4 +1,4 @@ -$File: COPYING,v 1.1 2008/02/05 19:08:11 christos Exp $ +$File: COPYING,v 1.2 2018/09/09 20:33:28 christos Exp $ Copyright (c) Ian F. Darwin 1986, 1987, 1989, 1990, 1991, 1992, 1994, 1995. Software written by Ian F. Darwin and others; maintained 1994- Christos Zoulas. @@ -15,7 +15,7 @@ are met: 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - + THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE diff --git a/ChangeLog b/ChangeLog index 4923bc3c163e..482a5f7c2de4 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,75 @@ +2019-05-14 22:26 Christos Zoulas <christos@zoulas.com> + + * release 5.37 + +2019-05-09 22:27 Christos Zoulas <christos@zoulas.com> + + * Make sure that continuation separators are printed + with -k within softmagic + +2019-05-06 22:27 Christos Zoulas <christos@zoulas.com> + + * Change SIGPIPE saving and restoring during compression to use + sigaction(2) instead of signal(3) and cache it. (Denys Vlasenko) + * Cache stat(2) calls more to reduce number of calls (Denys Vlasenko) + +2019-05-06 17:25 Christos Zoulas <christos@zoulas.com> + + * PR/77: Handle --mime-type and -k correctly. + +2019-05-03 15:26 Christos Zoulas <christos@zoulas.com> + + * Switch decompression code to use vfork() because + tools like rpmdiff and rpmbuild call libmagic + with large process footprints (Denys Vlasenko) + +2019-04-07 14:05 Christos Zoulas <christos@zoulas.com> + + * PR/75: --enable-zlib, did not work. + +2019-02-27 11:54 Christos Zoulas <christos@zoulas.com> + + * Improve regex efficiency (Michael Schroeder) by: + 1. Prefixing regex searches with regular search + for keywords where possible + 2. Using memmem(3) where available + +2019-02-20 10:16 Christos Zoulas <christos@zoulas.com> + + * release 5.36 + +2019-02-19 15:30 Christos Zoulas <christos@zoulas.com> + + * Fix cast to use cast macros + * Add UCS-32 builtin detection (PR/61) reported by tmc + +2019-02-18 18:24 Christos Zoulas <christos@zoulas.com> + + * Fix stack read (PR/62) and write (PR/64) stack overflows + reported by spinpx + +2018-10-18 19:32 Christos Zoulas <christos@zoulas.com> + + * release 5.35 + +2018-09-10 20:38 Christos Zoulas <christos@zoulas.com> + + * Add FreeBSD ELF core file support (John Baldwin) + +2018-08-20 18:40 Christos Zoulas <christos@zoulas.com> + + * PR/30: Allow all parameter values to be set (don't treat 0 specially) + * handle default annotations on the softmagic match instead at the + end. + +2018-07-25 10:17 Christos Zoulas <christos@zoulas.com> + + * PR/23: Recognize JSON files + +2018-07-25 10:17 Christos Zoulas <christos@zoulas.com> + + * PR/18: file --mime-encoding should not print mime-type + 2018-07-25 8:50 Christos Zoulas <christos@zoulas.com> * release 5.34 @@ -14,11 +86,11 @@ * release 5.33 -2018-02-24 14:50 Christos Zoulas <christos@zoulas.com> +2018-02-24 14:50 Christos Zoulas <christos@zoulas.com> * extend the support for ${x?:} expansions for magic descriptions -2018-02-21 16:25 Christos Zoulas <christos@zoulas.com> +2018-02-21 16:25 Christos Zoulas <christos@zoulas.com> * add support for ${x?:} in mime types to handle pie binaries. @@ -90,7 +162,7 @@ * Add missing overflow check in der magic (Jonas Wagner) 2016-10-25 10:40 Christos Zoulas <christos@zoulas.com> - + * release 5.29 2016-10-24 11:20 Christos Zoulas <christos@zoulas.com> @@ -131,11 +203,11 @@ - set offset to 0 on failure. 2016-05-13 12:00 Christos Zoulas <christos@zoulas.com> - + * release 5.27 2016-04-18 9:35 Christos Zoulas <christos@zoulas.com> - + * Errors comparing DER entries or computing offsets are just indications of malformed non-DER files. Don't print them. @@ -144,7 +216,7 @@ * Put new bytes constant in the right file (not the generated one) 2016-04-16 18:34 Christos Zoulas <christos@zoulas.com> - + * release 5.26 2016-03-31 13:50 Christos Zoulas <christos@zoulas.com> @@ -190,7 +262,7 @@ * PR/492: compression forking was broken with magic_buffer. 2015-09-16 9:50 Christos Zoulas <christos@zoulas.com> - + * release 5.25 2015-09-11 13:25 Christos Zoulas <christos@zoulas.com> @@ -218,7 +290,7 @@ * release 5.23 2015-06-09 16:10 Christos Zoulas <christos@zoulas.com> - + * Fix issue with regex range for magic with offset * Always return true from mget with USE (success to mget not match indication). Fixes mime evaluation after USE magic @@ -271,12 +343,12 @@ * add indirect relative for TIFF/Exif 2014-12-16 18:10 Christos Zoulas <christos@zoulas.com> - + * restructure elf note printing to avoid repeated messages * add note limit, suggested by Alexander Cherepanov 2014-12-16 16:53 Christos Zoulas <christos@zoulas.com> - + * Bail out on partial pread()'s (Alexander Cherepanov) * Fix incorrect bounds check in file_printable (Alexander Cherepanov) @@ -287,7 +359,7 @@ more places for safety * in ELF, instead of "(uses dynamic libraries)" when PT_INTERP is present print the interpreter name. - + 2014-12-10 20:01 Christos Zoulas <christos@zoulas.com> * release 5.21 @@ -330,7 +402,7 @@ on a byte by byte basis, so that we don't get issues with locale's trying to interpret random byte streams as UTF-8 and having printf error out with EILSEQ. - + 2014-10-17 11:48 Christos Zoulas <christos@zoulas.com> * fix bounds in note reading (Francisco Alonso / Red Hat) @@ -361,7 +433,7 @@ * release 5.19 2014-06-09 9:04 Christos Zoulas <christos@zoulas.com> - + * Misc buffer overruns and missing buffer size tests in cdf parsing (Francisco Alonso, Jan Kaluza) @@ -391,7 +463,7 @@ 2014-05-04 14:55 Christos Zoulas <christos@zoulas.com> - * PR/351: Fix compilation of empty files + * PR/351: Fix compilation of empty files 2014-04-30 17:39 Christos Zoulas <christos@zoulas.com> @@ -495,7 +567,7 @@ * Don't treat magic as an error if offset was past EOF (Christoph Biedl) 2013-05-28 17:25 Christos Zoulas <christos@zoulas.com> - + * Fix spacing issues in softmagic and elf (Jan Kaluza) 2013-05-02 18:00 Christos Zoulas <christos@zoulas.com> @@ -551,7 +623,7 @@ * Make getline public so that file can link against it. Perhaps it is better to rename it, or hide it differently. Fixes builds on platforms that do not provide it. - + 2013-01-07 16:30 Christos Zoulas <christos@zoulas.com> * Add SuS d{,1,2,4,8}, u{,1,2,4,8} and document @@ -901,7 +973,7 @@ (Cheng Renquan) 2009-05-08 13:40 Christos Zoulas <christos@zoulas.com> - + * lint fixes and more from NetBSD 2009-05-06 10:25 Christos Zoulas <christos@zoulas.com> @@ -921,7 +993,7 @@ 2009-03-27 18:43 Christos Zoulas <christos@zoulas.com> - * don't print \012- separators in the same magic entry + * don't print \012- separators in the same magic entry if it consists of multiple magic printing lines. 2009-03-23 10:20 Christos Zoulas <christos@zoulas.com> @@ -1 +1 @@ -See ChangeLog.
\ No newline at end of file +See ChangeLog. @@ -1,6 +1,6 @@ -## README for file(1) Command ## +## README for file(1) Command and the libmagic(3) library ## - @(#) $File: README,v 1.54 2018/05/30 03:06:56 christos Exp $ + @(#) $File: README,v 1.57 2019/02/06 00:20:56 christos Exp $ Mailing List: file@astron.com Mailing List archives: http://mailman.astron.com/pipermail/file/ @@ -63,53 +63,54 @@ magic numbers assigned to all sorts of data files that are in reasonable circulation. Send your magic numbers, in magic(5) format please, to the maintainer, Christos Zoulas. -COPYING - read this first. -README - read this second (you are currently reading this file). +COPYING - read this first. +README - read this second (you are currently reading this file). INSTALL - read on how to install -src/apprentice.c - parses /etc/magic to learn magic -src/apptype.c - used for OS/2 specific application type magic -src/ascmagic.c - third & last set of tests, based on hardwired assumptions. -src/asctime_r.c - replacement for OS's that don't have it. -src/asprintf.c - replacement for OS's that don't have it. -src/asctime_r.c - replacement for OS's that don't have it. -src/asprintf.c - replacement for OS's that don't have it. +src/apprentice.c - parses /etc/magic to learn magic +src/apptype.c - used for OS/2 specific application type magic +src/ascmagic.c - third & last set of tests, based on hardwired assumptions. +src/asctime_r.c - replacement for OS's that don't have it. +src/asprintf.c - replacement for OS's that don't have it. +src/asctime_r.c - replacement for OS's that don't have it. +src/asprintf.c - replacement for OS's that don't have it. src/buffer.c - buffer handling functions. -src/cdf.[ch] - parser for Microsoft Compound Document Files -src/cdf_time.c - time converter for CDF. -src/compress.c - handles decompressing files to look inside. -src/ctime_r.c - replacement for OS's that don't have it. +src/cdf.[ch] - parser for Microsoft Compound Document Files +src/cdf_time.c - time converter for CDF. +src/compress.c - handles decompressing files to look inside. +src/ctime_r.c - replacement for OS's that don't have it. src/der.[ch] - parser for Distinguished Encoding Rules src/dprintf.c - replacement for OS's that don't have it. src/elfclass.h - common code for elf 32/64. -src/encoding.c - handles unicode encodings -src/file.c - the main program -src/file.h - header file +src/encoding.c - handles unicode encodings +src/file.c - the main program +src/file.h - header file src/file_opts.h - list of options -src/fmtcheck.c - replacement for OS's that don't have it. -src/fsmagic.c - first set of tests the program runs, based on filesystem info -src/funcs.c - utilility functions -src/getline.c - replacement for OS's that don't have it. -src/getopt_long.c - replacement for OS's that don't have it. -src/gmtime_r.c - replacement for OS's that don't have it. -src/is_tar.c, tar.h - knows about Tape ARchive format (courtesy John Gilmore). -src/localtime_r.c - replacement for OS's that don't have it. +src/fmtcheck.c - replacement for OS's that don't have it. +src/fsmagic.c - first set of tests the program runs, based on filesystem info +src/funcs.c - utilility functions +src/getline.c - replacement for OS's that don't have it. +src/getopt_long.c - replacement for OS's that don't have it. +src/gmtime_r.c - replacement for OS's that don't have it. +src/is_json.c - knows about JavaScript Object Notation format (RFC 8259). +src/is_tar.c, tar.h - knows about Tape ARchive format (courtesy John Gilmore). +src/localtime_r.c - replacement for OS's that don't have it. src/magic.h.in - source file for magic.h -src/mygetopt.h - replacement for OS's that don't have it. -src/magic.c - the libmagic api -src/names.h - header file for ascmagic.c -src/pread.c - replacement for OS's that don't have it. -src/print.c - print results, errors, warnings. -src/readcdf.c - CDF wrapper. -src/readelf.[ch] - Stand-alone elf parsing code. -src/softmagic.c - 2nd set of tests, based on /etc/magic -src/mygetopt.h - replacement for OS's that don't have it. -src/strcasestr.c - replacement for OS's that don't have it. -src/strlcat.c - replacement for OS's that don't have it. -src/strlcpy.c - replacement for OS's that don't have it. -src/strndup.c - replacement for OS's that don't have it. +src/mygetopt.h - replacement for OS's that don't have it. +src/magic.c - the libmagic api +src/names.h - header file for ascmagic.c +src/pread.c - replacement for OS's that don't have it. +src/print.c - print results, errors, warnings. +src/readcdf.c - CDF wrapper. +src/readelf.[ch] - Stand-alone elf parsing code. +src/softmagic.c - 2nd set of tests, based on /etc/magic +src/mygetopt.h - replacement for OS's that don't have it. +src/strcasestr.c - replacement for OS's that don't have it. +src/strlcat.c - replacement for OS's that don't have it. +src/strlcpy.c - replacement for OS's that don't have it. +src/strndup.c - replacement for OS's that don't have it. src/tar.h - tar file definitions -src/vasprintf.c - for systems that don't have it. -doc/file.man - man page for the command +src/vasprintf.c - for systems that don't have it. +doc/file.man - man page for the command doc/magic.man - man page for the magic file, courtesy Guy Harris. Install as magic.4 on USG and magic.5 on V7 or Berkeley; cf Makefile. diff --git a/config.h.in b/config.h.in index 3c8ffa3b0250..97ed41c0e2bf 100644 --- a/config.h.in +++ b/config.h.in @@ -83,12 +83,6 @@ /* Define to 1 if you have the `z' library (-lz). */ #undef HAVE_LIBZ -/* Define to 1 if you have the <limits.h> header file. */ -#undef HAVE_LIMITS_H - -/* Define to 1 if you have the <locale.h> header file. */ -#undef HAVE_LOCALE_H - /* Define to 1 if you have the `localtime_r' function. */ #undef HAVE_LOCALTIME_R @@ -98,6 +92,9 @@ /* Define to 1 if <wchar.h> declares mbstate_t. */ #undef HAVE_MBSTATE_T +/* Define to 1 if you have the `memmem' function. */ +#undef HAVE_MEMMEM + /* Define to 1 if you have the <memory.h> header file. */ #undef HAVE_MEMORY_H @@ -116,18 +113,9 @@ /* Define to 1 if you have the `pread' function. */ #undef HAVE_PREAD -/* Define to 1 if you have the `setlocale' function. */ -#undef HAVE_SETLOCALE - -/* Define to 1 if you have the <signal.h> header file. */ -#undef HAVE_SIGNAL_H - /* Have sig_t type */ #undef HAVE_SIG_T -/* Define to 1 if you have the <stddef.h> header file. */ -#undef HAVE_STDDEF_H - /* Define to 1 if you have the <stdint.h> header file. */ #undef HAVE_STDINT_H @@ -137,9 +125,6 @@ /* Define to 1 if you have the `strcasestr' function. */ #undef HAVE_STRCASESTR -/* Define to 1 if you have the `strerror' function. */ -#undef HAVE_STRERROR - /* Define to 1 if you have the <strings.h> header file. */ #undef HAVE_STRINGS_H @@ -158,9 +143,6 @@ /* Define to 1 if you have the `strtof' function. */ #undef HAVE_STRTOF -/* Define to 1 if you have the `strtoul' function. */ -#undef HAVE_STRTOUL - /* HAVE_STRUCT_OPTION */ #undef HAVE_STRUCT_OPTION @@ -182,6 +164,9 @@ /* Define to 1 if you have the <sys/stat.h> header file. */ #undef HAVE_SYS_STAT_H +/* Define to 1 if you have the <sys/sysmacros.h> header file. */ +#undef HAVE_SYS_SYSMACROS_H + /* Define to 1 if you have the <sys/time.h> header file. */ #undef HAVE_SYS_TIME_H diff --git a/configure b/configure index 436a9460a043..a26523061fd6 100755 --- a/configure +++ b/configure @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for file 5.34. +# Generated by GNU Autoconf 2.69 for file 5.37. # # Report bugs to <christos@astron.com>. # @@ -590,8 +590,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='file' PACKAGE_TARNAME='file' -PACKAGE_VERSION='5.34' -PACKAGE_STRING='file 5.34' +PACKAGE_VERSION='5.37' +PACKAGE_STRING='file 5.37' PACKAGE_BUGREPORT='christos@astron.com' PACKAGE_URL='' @@ -1329,7 +1329,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures file 5.34 to adapt to many kinds of systems. +\`configure' configures file 5.37 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1399,7 +1399,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of file 5.34:";; + short | recursive ) echo "Configuration of file 5.37:";; esac cat <<\_ACEOF @@ -1511,7 +1511,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -file configure 5.34 +file configure 5.37 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2167,7 +2167,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by file $as_me 5.34, which was +It was created by file $as_me 5.37, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -3033,7 +3033,7 @@ fi # Define the identity of the package. PACKAGE='file' - VERSION='5.34' + VERSION='5.37' cat >>confdefs.h <<_ACEOF @@ -12784,7 +12784,7 @@ $as_echo "#define HAVE_SYS_WAIT_H 1" >>confdefs.h fi -for ac_header in stdint.h fcntl.h locale.h stdint.h inttypes.h unistd.h +for ac_header in stdint.h fcntl.h inttypes.h unistd.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" @@ -12797,7 +12797,7 @@ fi done -for ac_header in stddef.h utime.h wchar.h wctype.h limits.h +for ac_header in utime.h wchar.h wctype.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" @@ -12810,7 +12810,7 @@ fi done -for ac_header in getopt.h err.h xlocale.h signal.h +for ac_header in getopt.h err.h xlocale.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" @@ -12823,7 +12823,7 @@ fi done -for ac_header in sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h +for ac_header in sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h sys/sysmacros.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" @@ -12850,10 +12850,7 @@ fi done fi -ac_fn_c_check_type "$LINENO" "sig_t" "ac_cv_type_sig_t" " -#ifdef HAVE_SIGNAL_H -#include <signal.h> -#endif +ac_fn_c_check_type "$LINENO" "sig_t" "ac_cv_type_sig_t" "#include <signal.h> " if test "x$ac_cv_type_sig_t" = xyes; then : @@ -12976,7 +12973,8 @@ _ACEOF fi -ac_fn_c_check_member "$LINENO" "struct tm" "tm_gmtoff" "ac_cv_member_struct_tm_tm_gmtoff" "$ac_includes_default" +ac_fn_c_check_member "$LINENO" "struct tm" "tm_gmtoff" "ac_cv_member_struct_tm_tm_gmtoff" "#include <time.h> +" if test "x$ac_cv_member_struct_tm_tm_gmtoff" = xyes; then : cat >>confdefs.h <<_ACEOF @@ -14229,7 +14227,7 @@ fi fi -for ac_func in strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale +for ac_func in strndup mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale memmem do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" @@ -14583,7 +14581,8 @@ if test "$enable_zlib" = "yes"; then if test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" != "yesyes"; then as_fn_error $? "zlib support requested but not found" "$LINENO" 5 fi -elif test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" = "yesyes"; then +fi +if test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" = "yesyes"; then $as_echo "#define ZLIBSUPPORT 1" >>confdefs.h @@ -15134,7 +15133,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by file $as_me 5.34, which was +This file was extended by file $as_me 5.37, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -15200,7 +15199,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -file config.status 5.34 +file config.status 5.37 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" diff --git a/configure.ac b/configure.ac index ab4f6fc21cf4..7da62aa26987 100644 --- a/configure.ac +++ b/configure.ac @@ -1,5 +1,5 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([file],[5.34],[christos@astron.com]) +AC_INIT([file],[5.37],[christos@astron.com]) AM_INIT_AUTOMAKE([subdir-objects foreign]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -35,12 +35,12 @@ fi], [ ]) AC_MSG_CHECKING(for zlib support) -AC_ARG_ENABLE(zlib, +AC_ARG_ENABLE([zlib], [AS_HELP_STRING([--disable-zlib], [disable zlib compression support @<:@default=auto@:>@])]) AC_MSG_RESULT($enable_zlib) AC_MSG_CHECKING(for libseccomp support) -AC_ARG_ENABLE(libseccomp, +AC_ARG_ENABLE([libseccomp], [AS_HELP_STRING([--disable-libseccomp], [disable libseccomp sandboxing @<:@default=auto@:>@])]) AC_MSG_RESULT($enable_libseccomp) @@ -90,17 +90,14 @@ dnl Checks for headers AC_HEADER_STDC AC_HEADER_MAJOR AC_HEADER_SYS_WAIT -AC_CHECK_HEADERS(stdint.h fcntl.h locale.h stdint.h inttypes.h unistd.h) -AC_CHECK_HEADERS(stddef.h utime.h wchar.h wctype.h limits.h) -AC_CHECK_HEADERS(getopt.h err.h xlocale.h signal.h) -AC_CHECK_HEADERS(sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h) +AC_CHECK_HEADERS(stdint.h fcntl.h inttypes.h unistd.h) +AC_CHECK_HEADERS(utime.h wchar.h wctype.h) +AC_CHECK_HEADERS(getopt.h err.h xlocale.h) +AC_CHECK_HEADERS(sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h sys/sysmacros.h) if test "$enable_zlib" != "no"; then AC_CHECK_HEADERS(zlib.h) fi -AC_CHECK_TYPE([sig_t],[AC_DEFINE([HAVE_SIG_T],1,[Have sig_t type])],,[ -#ifdef HAVE_SIGNAL_H -#include <signal.h> -#endif]) +AC_CHECK_TYPE([sig_t],[AC_DEFINE([HAVE_SIG_T],1,[Have sig_t type])],,[#include <signal.h>]) dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST @@ -108,7 +105,7 @@ AC_TYPE_OFF_T AC_TYPE_SIZE_T AC_CHECK_MEMBERS([struct stat.st_rdev]) -AC_CHECK_MEMBERS([struct tm.tm_gmtoff]) +AC_CHECK_MEMBERS([struct tm.tm_gmtoff],,,[#include <time.h>]) AC_STRUCT_TIMEZONE AC_STRUCT_TIMEZONE_DAYLIGHT AC_SYS_LARGEFILE @@ -154,7 +151,7 @@ else fi]) dnl Checks for functions -AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale) +AC_CHECK_FUNCS(strndup mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale memmem) dnl Provide implementation of some required functions if necessary AC_REPLACE_FUNCS(getopt_long asprintf vasprintf strlcpy strlcat getline ctime_r asctime_r localtime_r gmtime_r pread strcasestr fmtcheck dprintf) @@ -178,7 +175,8 @@ if test "$enable_zlib" = "yes"; then if test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" != "yesyes"; then AC_MSG_ERROR([zlib support requested but not found]) fi -elif test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" = "yesyes"; then +fi +if test "$ac_cv_header_zlib_h$ac_cv_lib_z_gzopen" = "yesyes"; then AC_DEFINE([ZLIBSUPPORT], 1, [Enable zlib compression support]) fi diff --git a/doc/file.man b/doc/file.man index 0759f5f09704..63e95f1365ff 100644 --- a/doc/file.man +++ b/doc/file.man @@ -1,5 +1,5 @@ -.\" $File: file.man,v 1.131 2018/07/24 21:33:56 christos Exp $ -.Dd July 25, 2018 +.\" $File: file.man,v 1.135 2019/03/03 02:32:40 christos Exp $ +.Dd February 18, 2019 .Dt FILE __CSECTION__ .Os .Sh NAME @@ -159,7 +159,7 @@ two groups, so they are performed last. The language test routines also test for some miscellany (such as .Xr tar 1 -archives). +archives, JSON files). .Pp Any file that cannot be identified as having been written in any of the character sets listed above is simply said to be @@ -171,6 +171,8 @@ Causes the file command to output the file type and creator code as used by older MacOS versions. The code consists of eight letters, the first describing the file type, the latter the creator. +This option works properly only for file formats that have the +apple-style output defined. .It Fl b , Fl Fl brief Do not prepend filenames to output lines (brief mode). .It Fl C , Fl Fl compile @@ -213,6 +215,8 @@ Checks for, and looks inside, compressed files. .It elf Prints ELF file details, provided soft magic tests are enabled and the elf magic is found. +.It json +Examines JSON (RFC-7159) files by parsing them for compliance. .It soft Consults magic files. .It tar @@ -628,11 +632,11 @@ were written by John Gilmore from his public-domain program, and are not covered by the above license. .Sh BUGS Please report bugs and send patches to the bug tracker at -.Pa http://bugs.astron.com/ +.Pa https://bugs.astron.com/ or the mailing list at .Aq file@astron.com (visit -.Pa http://mailman.astron.com/mailman/listinfo/file +.Pa https://mailman.astron.com/mailman/listinfo/file first to subscribe). .Sh TODO Fix output so that tests for MIME and APPLE flags are not needed all diff --git a/doc/libmagic.man b/doc/libmagic.man index 4c7e42ff4f9c..086f0659f113 100644 --- a/doc/libmagic.man +++ b/doc/libmagic.man @@ -1,6 +1,6 @@ -.\" $File: libmagic.man,v 1.41 2017/05/23 21:54:07 christos Exp $ +.\" $File: libmagic.man,v 1.44 2018/09/09 20:33:28 christos Exp $ .\" -.\" Copyright (c) Christos Zoulas 2003. +.\" Copyright (c) Christos Zoulas 2003, 2018. .\" All Rights Reserved. .\" .\" Redistribution and use in source and binary forms, with or without @@ -25,7 +25,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd May 23, 2017 +.Dd August 18, 2018 .Dt LIBMAGIC 3 .Os .Sh NAME @@ -159,6 +159,8 @@ Don't examine tar files. Don't check for various types of text files. .It Dv MAGIC_NO_CHECK_TOKENS Don't look for known tokens inside ascii files. +.It Dv MAGIC_NO_CHECK_JSON +Don't example JSON files. .El .Pp The @@ -391,6 +393,19 @@ The compiled default magic database. .Sh SEE ALSO .Xr file __CSECTION__ , .Xr magic __FSECTION__ +.Sh BUGS +The results from +.Fn magic_buffer +and +.Fn magic_file +where the buffer and the file contain the same data +can produce different results, because in the +.Fn magic_file +case, the program can +.Xr lseek 2 +and +.Xr stat 2 +the file descriptor. .Sh AUTHORS .An M\(oans Rullg\(oard Initial libmagic implementation, and configuration. diff --git a/doc/magic.man b/doc/magic.man index 15165daa5eab..bc69604a3df5 100644 --- a/doc/magic.man +++ b/doc/magic.man @@ -1,6 +1,5 @@ -.It S2 -.\" $File: magic.man,v 1.93 2018/06/22 20:39:49 christos Exp $ -.Dd June 22, 2018 +.\" $File: magic.man,v 1.96 2019/01/21 14:56:53 christos Exp $ +.Dd January 21, 2019 .Dt MAGIC __FSECTION__ .Os .\" install as magic.4 on USG, magic.5 on V7, Berkeley and Linux systems. @@ -117,13 +116,13 @@ The following modifiers are supported: .It B A byte length (default). .It H -A 4 byte big endian length. -.It h A 2 byte big endian length. +.It h +A 2 byte little endian length. .It L -A 4 byte little endian length. +A 4 byte big endian length. .It l -A 2 byte little endian length. +A 4 byte little endian length. .It J The length includes itself in its count. .El diff --git a/magic/Header b/magic/Header index 831122e27a80..345a50fb1e67 100644 --- a/magic/Header +++ b/magic/Header @@ -2,4 +2,4 @@ # Format is described in magic(files), where: # files is 5 on V7 and BSD, 4 on SV, and ?? on SVID. # Don't edit this file, edit /etc/magic or send your magic improvements -# to the maintainers, at file@mx.gw.com +# to the maintainers, at file@astron.com diff --git a/magic/Magdir/acorn b/magic/Magdir/acorn index c4debd0849fb..4aa34551a74b 100644 --- a/magic/Magdir/acorn +++ b/magic/Magdir/acorn @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: acorn,v 1.6 2017/10/19 16:40:37 christos Exp $ +# $File: acorn,v 1.7 2019/04/19 00:42:27 christos Exp $ # acorn: file(1) magic for files found on Acorn systems # @@ -83,7 +83,7 @@ # compression mode y (0 - 4) for GIF LZW with a maximum n bits # (y~n,0~12,1~13,2~14,3~15,4~16) >>>5 ulelong+12 x \b, LZW %u-bits compression -# http://www.filebase.org.uk/filetypes +# https://www.filebase.org.uk/filetypes # !Packdir compressed archive has three hexadecimal digits code 68E !:mime application/x-acorn-68E !:ext pkd/bin diff --git a/magic/Magdir/adventure b/magic/Magdir/adventure index 6fae85adfd40..bd7f863be28b 100644 --- a/magic/Magdir/adventure +++ b/magic/Magdir/adventure @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: adventure,v 1.17 2017/07/03 16:03:40 christos Exp $ +# $File: adventure,v 1.18 2019/04/19 00:42:27 christos Exp $ # adventure: file(1) magic for Adventure game files # # from Allen Garvin <earendil@faeryland.tamu-commerce.edu> @@ -21,8 +21,8 @@ # Updated by Adam Buchbinder <adam.buchbinder@gmail.com> # #http://www.gnelson.demon.co.uk/zspec/sect11.html -#http://www.jczorkmid.net/~jpenney/ZSpec11-latest.txt -#http://en.wikipedia.org/wiki/Z-machine +#https://www.jczorkmid.net/~jpenney/ZSpec11-latest.txt +#https://en.wikipedia.org/wiki/Z-machine # The first byte is the Z-machine revision; it is always between 1 and 8. We # had false matches (for instance, inbig5.ocp from the Omega TeX extension as # well as an occasional MP3 file), so we sanity-check the version number. @@ -111,7 +111,7 @@ # Danny Milosavljevic <danny.milo@gmx.net> # These are ADRIFT (adventure game standard) game files, extension .taf # Checked from source at (http://www.adrift.co/) and various taf files -# found at the Interactive Fiction Archive (http://ifarchive.org/) +# found at the Interactive Fiction Archive (https://ifarchive.org/) 0 belong 0x3C423FC9 >4 belong 0x6A87C2CF Adrift game file version >>8 belong 0x94453661 3.80 diff --git a/magic/Magdir/algol68 b/magic/Magdir/algol68 index a9aad46e1fc3..3675b840e3b6 100644 --- a/magic/Magdir/algol68 +++ b/magic/Magdir/algol68 @@ -1,17 +1,17 @@ #------------------------------------------------------------------------------ -# $File: algol68,v 1.2 2016/10/17 14:17:48 christos Exp $ +# $File: algol68,v 1.3 2018/10/19 01:04:21 christos Exp $ # algol68: file(1) magic for Algol 68 source # 0 search/8192 (input, Algol 68 source text !:mime text/x-Algol68 -0 regex \^PROC Algol 68 source text +0 regex/1024 \^PROC Algol 68 source text !:mime text/x-Algol68 -0 regex MODE[\t\ ] Algol 68 source text +0 regex/1024 \bMODE[\t\ ] Algol 68 source text !:mime text/x-Algol68 -0 regex REF[\t\ ] Algol 68 source text +0 regex/1024 \bREF[\t\ ] Algol 68 source text !:mime text/x-Algol68 -0 regex FLEX[\t\ ]\*\\[ Algol 68 source text +0 regex/1024 \bFLEX[\t\ ]\*\\[ Algol 68 source text !:mime text/x-Algol68 #0 regex [\t\ ]OD Algol 68 source text #!:mime text/x-Algol68 diff --git a/magic/Magdir/amigaos b/magic/Magdir/amigaos index d9330bd1493c..e719921ef39f 100644 --- a/magic/Magdir/amigaos +++ b/magic/Magdir/amigaos @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: amigaos,v 1.16 2017/03/17 21:35:28 christos Exp $ +# $File: amigaos,v 1.17 2018/10/16 18:57:19 christos Exp $ # amigaos: file(1) magic for AmigaOS binary formats: # @@ -66,3 +66,22 @@ # From: Przemek Kramarczyk <pkramarczyk@gmail.com> 0 string .KEY AmigaDOS script 0 string .key AmigaDOS script + +# AMOS Basic file formats +# https://www.exotica.org.uk/wiki/AMOS_file_formats +0 string AMOS\040Basic\040 AMOS Basic source code +>11 byte =0x56 \b, tested +>11 byte =0x76 \b, untested +0 string AMOS\040Pro AMOS Basic source code +>11 byte =0x56 \b, tested +>11 byte =0x76 \b, untested +0 string AmSp AMOS Basic sprite bank +>4 beshort x \b, %d sprites +0 string AmIc AMOS Basic icon bank +>4 beshort x \b, %d icons +0 string AmBk AMOS Basic memory bank +>4 beshort x \b, bank number %d +>8 belong&0xFFFFFFF x \b, length %d +>12 regex .{8} \b, type %s +0 string AmBs AMOS Basic memory banks +>4 beshort x \b, %d banks diff --git a/magic/Magdir/android b/magic/Magdir/android index dca5c33482bf..a9cfb3575a1b 100644 --- a/magic/Magdir/android +++ b/magic/Magdir/android @@ -1,6 +1,6 @@ #------------------------------------------------------------ -# $File: android,v 1.10 2017/03/17 21:35:28 christos Exp $ +# $File: android,v 1.12 2019/04/19 00:42:27 christos Exp $ # Various android related magic entries #------------------------------------------------------------ @@ -32,33 +32,68 @@ # Android Backup archive # From: Ariel Shkedi -# File extension: .ab -# No mime-type defined +# Update: Joerg Jenderek # URL: https://github.com/android/platform_frameworks_base/blob/\ # 0bacfd2ba68d21a68a3df345b830bc2a1e515b5a/services/java/com/\ # android/server/BackupManagerService.java#L2367 +# Reference: https://sourceforge.net/projects/adbextractor/ +# android-backup-extractor/perl/backupencrypt.pl +# Note: only unix line feeds "\n" found # After the header comes a tar file # If compressed, the entire tar file is compressed with JAVA deflate # # Include the version number hardcoded with the magic string to avoid # false positives -0 string/b ANDROID\ BACKUP\n1\n Android Backup +0 string/b ANDROID\ BACKUP\n Android Backup +# maybe look for some more characteristics like linefeed '\n' or version +#>16 string \n +# No mime-type defined offically +!:mime application/x-google-ab +!:ext ab +# on 2nd line version (often 1, 2 on kitkat 4.4.3+, 4 on 7.1.2) +>15 string >\0 \b, version %s +# "1" on 3rd line means compressed >17 string 0\n \b, Not-Compressed >17 string 1\n \b, Compressed +# The 4th line is encryption "none" or "AES-256" # any string as long as it's not the word none (which is matched below) +>19 string none\n \b, Not-Encrypted +# look for backup content after line with encryption info +#>>19 search/7 \n +# data part after header for not encrypted Android Backup +#>>>&0 ubequad x \b, content 0x%16.16llx... +# look for zlib compressed by ./compress after message with 1 space at end +#>>>&0 indirect x \b; contains +# look for tar archive block by ./archive for package name manifest +>>288 string ustar \b; contains +>>>31 use tar-file +# look for zip/jar archive by ./archive ./zip after message with 1 space at end +#>>2079 search/1025/s PK\003\004 \b; contains +#>>>&0 indirect x +>19 string !none >>19 regex/1l \^([^n\n]|n[^o]|no[^n]|non[^e]|none.+).* \b, Encrypted (%s) ->>19 string none\n \b, Not-Encrypted # Commented out because they don't seem useful to print # (but they are part of the header - the tar file comes after them): +# The 5th line is User Password Salt (128 Hex) +# string length too high with standard src configuration +#>>>&1 string >\0 \b, PASSWORD salt: "%-128.128s" #>>>&1 regex/1l .* \b, Password salt: %s +# The 6th line is Master Key Checksum Salt (128 Hex) #>>>>&1 regex/1l .* \b, Master salt: %s +# The 7th line is Number of PBDKF2 Rounds (10000) #>>>>>&1 regex/1l .* \b, PBKDF2 rounds: %s +# The 8th line is User key Initialization Vector (IV) (32 Hex) #>>>>>>&1 regex/1l .* \b, IV: %s +#>>>>>>&1 regex/1l .* \b, IV: %s +# The 9th line is Master IV+Key+Checksum (192 Hex) #>>>>>>>&1 regex/1l .* \b, Key: %s +# look for new line separator char after line number 9 +#>>>0x204 ubyte 0x0a NL found +#>>>>&1 ubequad x \b, Content magic %16.16llx # *.pit files by Joerg Jenderek -# http://forum.xda-developers.com/showthread.php?p=9122369 -# http://forum.xda-developers.com/showthread.php?t=816449 +# https://forum.xda-developers.com/showthread.php?p=9122369 +# https://forum.xda-developers.com/showthread.php?t=816449 # Partition Information Table for Samsung's smartphone with Android # used by flash software Odin 0 ulelong 0x12349876 diff --git a/magic/Magdir/animation b/magic/Magdir/animation index 76e45de57e89..aaf32dd4c43c 100644 --- a/magic/Magdir/animation +++ b/magic/Magdir/animation @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: animation,v 1.68 2018/05/06 16:08:07 christos Exp $ +# $File: animation,v 1.71 2019/04/19 00:42:27 christos Exp $ # animation: file(1) magic for animation/movie formats # # animation formats @@ -32,9 +32,9 @@ !:mime application/x-quicktime-player 4 string/W jP JPEG 2000 image !:mime image/jp2 -# http://www.ftyps.com/ with local additions +# https://www.ftyps.com/ with local additions 4 string ftyp ISO Media -# http://aeroquartet.com/wordpress/2016/03/05/3-xavc-s/ +# https://aeroquartet.com/wordpress/2016/03/05/3-xavc-s/ >8 string XAVC \b, MPEG v4 system, Sony XAVC Codec >>96 string x \b, Audio "%.4s" >>118 beshort x at %dHz @@ -46,7 +46,7 @@ >>11 byte 4 \b v4 (H.263/AMR GSM 6.10) >>11 byte 5 \b v5 (H.263/AMR GSM 6.10) >>11 byte 6 \b v6 (ITU H.264/AMR GSM 6.10) -# http://www.3gpp2.org/Public_html/Specs/C.S0050-B_v1.0_070521.pdf +# https://www.3gpp2.org/Public_html/Specs/C.S0050-B_v1.0_070521.pdf # Section 8.1.1, corresponds to a, b, c >>11 byte 0x61 \b C.S0050-0 V1.0 >>11 byte 0x62 \b C.S0050-0-A V1.0.0 @@ -855,6 +855,7 @@ # conti_count 4 bit - 0 belong&0xFF5FFF10 0x47400010 >188 byte 0x47 MPEG transport stream data +!:mime video/MP2T # DIF digital video file format <mpruett@sgi.com> 0 belong&0xffffff00 0x1f070000 DIF @@ -892,10 +893,10 @@ 0 string/w #VRML\ V2.0\ utf8 ISO/IEC 14772 VRML 97 file !:mime model/vrml -# X3D (Extensible 3D) [http://www.web3d.org/specifications/x3d-3.0.dtd] +# X3D (Extensible 3D) [https://www.web3d.org/specifications/x3d-3.0.dtd] # From Michel Briand <michelbriand@free.fr> # mimetype from https://www.iana.org/assignments/media-types/model/x3d+xml -# Example http://www.web3d.org/x3d/content/examples/Basic/course/CreateX3DFromStringRandomSpheres.x3d +# Example https://www.web3d.org/x3d/content/examples/Basic/course/CreateX3DFromStringRandomSpheres.x3d 0 string/w \<?xml\ version= !:strength + 5 >20 search/1000/w \<!DOCTYPE\ X3D X3D (Extensible 3D) model xml text @@ -967,7 +968,7 @@ # Type: Bink Video # Extension: .bik -# URL: http://wiki.multimedia.cx/index.php?title=Bink_Container +# URL: https://wiki.multimedia.cx/index.php?title=Bink_Container # From: <hoehle@users.sourceforge.net> 2008-07-18 0 string BIK Bink Video >3 regex =[a-z] rev.%s @@ -988,61 +989,61 @@ #>>51 byte&0x10 !0 DCT # Type: NUT Container -# URL: http://wiki.multimedia.cx/index.php?title=NUT +# URL: https://wiki.multimedia.cx/index.php?title=NUT # From: Adam Buchbinder <adam.buchbinder@gmail.com> 0 string nut/multimedia\ container\0 NUT multimedia container # Type: Nullsoft Video (NSV) -# URL: http://wiki.multimedia.cx/index.php?title=Nullsoft_Video +# URL: https://wiki.multimedia.cx/index.php?title=Nullsoft_Video # From: Mike Melanson <mike@multimedia.cx> 0 string NSVf Nullsoft Video # Type: REDCode Video -# URL: http://www.red.com/ ; http://wiki.multimedia.cx/index.php?title=REDCode +# URL: https://www.red.com/ ; https://wiki.multimedia.cx/index.php?title=REDCode # From: Mike Melanson <mike@multimedia.cx> 4 string RED1 REDCode Video # Type: MTV Multimedia File -# URL: http://wiki.multimedia.cx/index.php?title=MTV +# URL: https://wiki.multimedia.cx/index.php?title=MTV # From: Mike Melanson <mike@multimedia.cx> 0 string AMVS MTV Multimedia File # Type: ARMovie -# URL: http://wiki.multimedia.cx/index.php?title=ARMovie +# URL: https://wiki.multimedia.cx/index.php?title=ARMovie # From: Mike Melanson <mike@multimedia.cx> 0 string ARMovie\012 ARMovie # Type: Interplay MVE Movie -# URL: http://wiki.multimedia.cx/index.php?title=Interplay_MVE +# URL: https://wiki.multimedia.cx/index.php?title=Interplay_MVE # From: Mike Melanson <mike@multimedia.cx> 0 string Interplay\040MVE\040File\032 Interplay MVE Movie # Type: Windows Television DVR File -# URL: http://wiki.multimedia.cx/index.php?title=WTV +# URL: https://wiki.multimedia.cx/index.php?title=WTV # From: Mike Melanson <mike@mutlimedia.cx> # This takes the form of a Windows-style GUID 0 bequad 0xB7D800203749DA11 >8 bequad 0xA64E0007E95EAD8D Windows Television DVR Media # Type: Sega FILM/CPK Multimedia -# URL: http://wiki.multimedia.cx/index.php?title=Sega_FILM +# URL: https://wiki.multimedia.cx/index.php?title=Sega_FILM # From: Mike Melanson <mike@multimedia.cx> 0 string FILM Sega FILM/CPK Multimedia, >32 belong x %d x >28 belong x %d # Type: Nintendo THP Multimedia -# URL: http://wiki.multimedia.cx/index.php?title=THP +# URL: https://wiki.multimedia.cx/index.php?title=THP # From: Mike Melanson <mike@multimedia.cx> 0 string THP\0 Nintendo THP Multimedia # Type: BBC Dirac Video -# URL: http://wiki.multimedia.cx/index.php?title=Dirac +# URL: https://wiki.multimedia.cx/index.php?title=Dirac # From: Mike Melanson <mike@multimedia.cx> 0 string BBCD BBC Dirac Video # Type: RAD Game Tools Smacker Multimedia -# URL: http://wiki.multimedia.cx/index.php?title=Smacker +# URL: https://wiki.multimedia.cx/index.php?title=Smacker # From: Mike Melanson <mike@multimedia.cx> 0 string SMK RAD Game Tools Smacker Multimedia >3 byte x version %c, @@ -1064,3 +1065,10 @@ >8 string AHDR LucasArts Smush Animation Format (SAN) video 0 string SANM >8 string SHDR LucasArts Smush v2 (SANM) video + +# Type: Scaleform video +# Extension: .usm +# URL: https://wiki.multimedia.cx/index.php/USM +# From: David Korth <gerbilsoft@gerbilsoft.com> +0 string CRID +>32 string @UTF Scaleform video diff --git a/magic/Magdir/apple b/magic/Magdir/apple index 39e838b57bd2..4ac10fc5be92 100644 --- a/magic/Magdir/apple +++ b/magic/Magdir/apple @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: apple,v 1.39 2018/03/02 15:26:39 christos Exp $ +# $File: apple,v 1.43 2019/04/19 00:42:27 christos Exp $ # apple: file(1) magic for Apple file formats # 0 search/1/t FiLeStArTfIlEsTaRt binscii (apple ][) text @@ -11,22 +11,97 @@ 0 belong 0x00051600 AppleSingle encoded Macintosh file 0 belong 0x00051607 AppleDouble encoded Macintosh file +# Type: Apple Emulator WOZ format +# From: Greg Wildman <greg@apple2.org.za> +# Ref: https://applesaucefdc.com/woz/reference/ +# Ref: https://applesaucefdc.com/woz/reference2/ +# +# Note: The following test are mostly identical. I would rather not +# use a regex to identify the WOZ format number. +0 string WOZ1 +>4 string \xFF\x0A\x0D\x0A Apple ][ WOZ 1.0 Disk Image +>12 string INFO +>>21 byte 01 \b, 5.25 inch +>>21 byte 02 \b, 3.5 inch +>>22 byte 01 \b, write protected +>>23 byte 01 \b, cross track synchronized +>>25 string/T x \b, %.32s +0 string WOZ2 +>4 string \xFF\x0A\x0D\x0A Apple ][ WOZ 2.0 Disk Image +>12 string INFO +>>21 byte 01 \b, 5.25 inch +>>21 byte 02 \b, 3.5 inch +>>22 byte 01 \b, write protected +>>23 byte 01 \b, cross track synchronized +>>25 string/T x \b, %.32s + +# Type: Apple Emulator disk images +# From: Greg Wildman <greg@apple2.org.za> +# ProDOS boot loader? +0 string \x01\x38\xB0\x03\x4C Apple ProDOS Image +# Detect Volume Directory block ($02) +>0x400 string \x00\x00\x03\x00 +>>0x404 byte &0xF0 +>>>0x405 string x \b, Volume /%s +>>>0x429 leshort x \b, %u Blocks +# ProDOS ordered ? +>0xb00 string \x00\x00\x03\x00 +>>0xb04 byte &0xF0 +>>>0xb05 string x \b, Volume /%s +>>>0xb29 leshort x \b, %u Blocks +# +# DOS3.3 boot loader? +0 string \x01\xA5\x27\xC9\x09\xD0\x18\xA5\x2B +>0x11001 string \x11\x0F\x03 Apple DOS 3.3 Image +>>0x11006 byte x \b, Volume %u +>>0x11034 byte x \b, %u Tracks +>>0x11035 byte x \b, %u Sectors +>>0x11036 leshort x \b, %u bytes per sector +# DOS3.2 ? +>0x11001 string \x11\x0C\x02 Apple DOS 3.2 Image +>>0x11006 byte x \b, Volume %u +>>0x11034 byte x \b, %u Tracks +>>0x11035 byte x \b, %u Sectors +>>0x11036 leshort x \b, %u bytes per sector +# DOS3.1 ? +>0x11001 string \x11\x0C\x01 +>>0x11c00 string \x00\x11\x0B Apple DOS 3.1 Image +# +# Pascal boot loader? +0 string \x01\xE0\x60\xF0\x03\x4C\xE3\x08\xAD +>0xd6 pstring SYSTEM.APPLE +>>0xb00 leshort 0x0000 +>>>0xb04 leshort 0x0000 Apple Pascal Image +>>>>0xb06 pstring x \b, Volume %s: +>>>>0xb0e leshort x \b, %u Blocks +>>>>0xb10 leshort x \b, %u Files + # Type: Apple Emulator 2IMG format # From: Radek Vokal <rvokal@redhat.com> -0 string 2IMG Apple ][ 2IMG Disk Image ->4 string XGS! \b, XGS ->4 string CTKG \b, Catakig ->4 string ShIm \b, Sheppy's ImageMaker ->4 string WOOF \b, Sweet 16 ->4 string B2TR \b, Bernie ][ the Rescue ->4 string !nfc \b, ASIMOV2 ->4 string x \b, Unknown Format ->0xc byte 00 \b, DOS 3.3 sector order ->>0x10 byte 00 \b, Volume 254 ->>0x10 byte&0x7f x \b, Volume %u ->0xc byte 01 \b, ProDOS sector order ->>0x14 short x \b, %u Blocks ->0xc byte 02 \b, NIB data +# Update: Greg Wildman <greg@apple2.org.za> +0 string 2IMG Apple ][ 2IMG Disk Image +>4 clear x +>4 string XGS! \b, XGS +>4 string CTKG \b, Catakig +>4 string ShIm \b, Sheppy's ImageMaker +>4 string SHEP \b, Sheppy's ImageMaker +>4 string WOOF \b, Sweet 16 +>4 string B2TR \b, Bernie ][ the Rescue +>4 string \!nfc \b, ASIMOV2 +>4 string \>BD\< \b, Brutal Deluxe's Cadius +>4 string CdrP \b, CiderPress +>4 string Vi][ \b, Virtual ][ +>4 string PRFS \b, ProFUSE +>4 string FISH \b, FishWings +>4 string RVLW \b, Revival for Windows +>4 default x +>>4 string x \b, Creator tag "%-4.4s" +>0xc byte 00 \b, DOS 3.3 sector order +>>0x10 byte 00 \b, Volume 254 +>>0x10 byte&0x7f x \b, Volume %u +>0xc byte 01 \b, ProDOS sector order +>>0x14 short x \b, %u Blocks +>0xc byte 02 \b, NIB data # magic for Newton PDA package formats # from Ruda Moura <ruda@helllabs.org> @@ -88,7 +163,7 @@ # http://home.earthlink.net/~hughhood/appleiiworksenvoy/ # ('p' + 1-byte ProDOS File Type + 2-byte ProDOS Aux Type') # $70 $1A $F8 $FF is this the apple type ? -#:apple pdospøÿ +#:apple pdosp^Z\xf8\xff !:ext awp # minimum version needed to read this files. SFMinVers (0 , 30~3.0 ) >>>183 ubyte 30 3.0 @@ -354,7 +429,7 @@ >>6 ubeshort x \b, type 0x%x # URL: https://en.wikipedia.org/wiki/Apple_Partition_Map -# Reference: http://opensource.apple.com/source/IOStorageFamily/IOStorageFamily-116/IOApplePartitionScheme.h +# Reference: https://opensource.apple.com/source/IOStorageFamily/IOStorageFamily-116/IOApplePartitionScheme.h # Update: Joerg Jenderek # Yes, the 3rd and 4th bytes pmSigPad are reserved, but we use them to make the # magic stronger. @@ -396,8 +471,8 @@ # is the startup partition APPLE_PS_STARTUP >>88 ubelong &0x80000000 \b, is the startup partition -#http://wiki.mozilla.org/DS_Store_File_Format -#http://en.wikipedia.org/wiki/.DS_Store +#https://wiki.mozilla.org/DS_Store_File_Format +#https://en.wikipedia.org/wiki/.DS_Store 0 string \0\0\0\1Bud1\0 Apple Desktop Services Store # HFS/HFS+ Resource fork files (andrew.roazen@nau.edu Apr 13 2015) diff --git a/magic/Magdir/archive b/magic/Magdir/archive index f8cf3a2684b5..cd0213fa9f3c 100644 --- a/magic/Magdir/archive +++ b/magic/Magdir/archive @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: archive,v 1.119 2018/04/24 23:19:45 christos Exp $ +# $File: archive,v 1.129 2019/05/09 18:58:02 christos Exp $ # archive: file(1) magic for archive formats (see also "msdos" for self- # extracting compressed archives) # @@ -148,7 +148,7 @@ >>>257 string >\0 \b, comment: %-.40s # Incremental snapshot gnu-tar format from: -# http://www.gnu.org/software/tar/manual/html_node/Snapshot-Files.html +# https://www.gnu.org/software/tar/manual/html_node/Snapshot-Files.html 0 string GNU\ tar- GNU tar incremental snapshot data >&0 regex [0-9]\.[0-9]+-[0-9]+ version %s @@ -226,21 +226,48 @@ # a portable archive whose first member has a name beginning with # "debian". # +# Update: Joerg Jenderek +# URL: https://en.wikipedia.org/wiki/Deb_(file_format) 0 string =!<arch>\ndebian ->8 string debian-split part of multipart Debian package +# https://manpages.debian.org/testing/dpkg/dpkg-split.1.en.html +>14 string -split part of multipart Debian package !:mime application/vnd.debian.binary-package ->8 string debian-binary Debian binary package +# udeb is used for stripped down deb file +!:ext deb/udeb +>14 string -binary Debian binary package !:mime application/vnd.debian.binary-package ->8 string !debian +!:ext deb/udeb +# This should not happen +>14 default x Unknown Debian package +# NL terminated version; for most Debian cases this is 2.0 or 2.1 for splitted >68 string >\0 (format %s) -# These next two lines do not work, because a bzip2 Debian archive -# still uses gzip for the control.tar (first in the archive). Only -# data.tar varies, and the location of its filename varies too. -# file/libmagic does not current have support for ascii-string based -# (offsets) as of 2005-09-15. -#>81 string bz2 \b, uses bzip2 compression -#>84 string gz \b, uses gzip compression -#>136 ledate x created: %s +#>68 string !2.0\n +#>>68 string x (format %.3s) +>68 string =2.0\n +# 2nd archive name=control archive name like control.tar.gz or control.tar.xz +>>72 string >\0 \b, with %.14s +# look for 3rd archive name=data archive name like data.tar.{gz,xz,bz2,lzma} +>>0 search/0x93e4f data.tar. \b, data compression +# the above line only works if FILE_BYTES_MAX in ../../src/file.h is raised +# for example like libreoffice-dev-doc_1%3a5.2.7-1+rpi1+deb9u3_all.deb +>>>&0 string x %.4s +# splitted debian package case +>68 string =2.1\n +# dpkg-1.18.25/dpkg-split/info.c +# NL terminated ASCII package name like ckermit +>>&0 string x \b, %s +# NL terminated package version like 302-5.3 +>>>&1 string x %s +# NL terminated MD5 checksum +>>>>&1 string x \b, MD5 %s +# NL terminated original package length +>>>>>&1 string x \b, unsplitted size %s +# NL terminated part length +>>>>>>&1 string x \b, part lenght %s +# NL terminated package part like n/m +>>>>>>>&1 string x \b, part %s +# NL terminated package architecture like armhf since dpkg 1.16.1 or later +>>>>>>>>&1 string x \b, %s # # MIPS archive; they're in the portable archive format, and need to go @@ -257,15 +284,35 @@ >19 string B and an EB hash table >22 string X -- out of date -0 search/1 -h- Software Tools format archive text - # # BSD/SVR2-and-later portable archive formats. # +# Update: Joerg Jenderek +# URL: http://fileformats.archiveteam.org/wiki/AR +# Reference: https://www.unix.com/man-page/opensolaris/3HEAD/ar.h/ +# Note: Mach-O universal binary in ./cafebabe is dependent +# TODO: unify current ar archive, MIPS archive, Debian package +# distinguish BSD, SVR; 32, 64 bit; HP from other 32-bit SVR; +# *.ar packages from *.a libraries. handle empty archive 0 string =!<arch>\n current ar archive +# print first and possibly second ar_name[16] for debugging purpose +#>8 string x \b, 1st "%.16s" +#>68 string x \b, 2nd "%.16s" !:mime application/x-archive +# a in most case for libraries; lib for Microsoft libraries; ar else cases +!:ext a/lib/ar >8 string __.SYMDEF random library +# first member with long marked name __.SYMDEF SORTED implies BSD library >68 string __.SYMDEF\ SORTED random library +# Reference: https://parisc.wiki.kernel.org/images-parisc/b/b2/Rad_11_0_32.pdf +# "archive file" entry moved from ./hp +# LST header system_id 0210h~PA-RISC 1.1,... identifies the target architecture +# LST header a_magic 0619h~relocatable library +>68 belong 0x020b0619 - PA-RISC1.0 relocatable library +>68 belong 0x02100619 - PA-RISC1.1 relocatable library +>68 belong 0x02110619 - PA-RISC1.2 relocatable library +>68 belong 0x02140619 - PA-RISC2.0 relocatable library +#EOF for common ar archives # # "Thin" archive, as can be produced by GNU ar. @@ -275,6 +322,8 @@ >68 belong 1 %d symbol entry >68 belong >1 %d symbol entries +0 search/1 -h- Software Tools format archive text + # ARC archiver, from Daniel Quinlan (quinlan@yggdrasil.com) # # The first byte is the magic (0x1a), byte 2 is the compression type for @@ -799,7 +848,7 @@ # LHARC/LHA archiver (Greg Roelofs, newt@uchicago.edu) # Update: Joerg Jenderek # URL: https://en.wikipedia.org/wiki/LHA_(file_format) -# Reference: http://web.archive.org/web/20021005080911/http://www.osirusoft.com/joejared/lzhformat.html +# Reference: https://web.archive.org/web/20021005080911/http://www.osirusoft.com/joejared/lzhformat.html # # check and display information of lharc (LHa,PMarc) file 0 name lharc-file @@ -999,7 +1048,7 @@ !:ext rar # Very old RAR archive -# http://jasonblanks.com/wp-includes/images/papers/KnowyourarchiveRAR.pdf +# https://jasonblanks.com/wp-includes/images/papers/KnowyourarchiveRAR.pdf 0 string RE\x7e\x5e RAR archive data (<v1.5) !:mime application/x-rar !:ext rar/cbr @@ -1020,7 +1069,9 @@ 0 string PK\005\006 Zip archive data (empty) !:mime application/zip !:ext zip/cbz +!:strength +1 0 string PK\003\004 +!:strength +1 # Specialised zip formats which start with a member named 'mimetype' # (stored uncompressed, with no 'extra field') containing the file's MIME type. @@ -1063,7 +1114,7 @@ >>>62 string base Database file # OpenDocument formats (for OpenOffice 2.x / StarOffice >= 8) -# http://lists.oasis-open.org/archives/office/200505/msg00006.html +# https://lists.oasis-open.org/archives/office/200505/msg00006.html # (mimetype contains "application/vnd.oasis.opendocument.<SUBTYPE>") >>50 string vnd.oasis.opendocument. OpenDocument >>>73 string text @@ -1112,7 +1163,7 @@ !:mime application/vnd.oasis.opendocument.image-template # EPUB (OEBPS) books using OCF (OEBPS Container Format) -# http://www.idpf.org/ocf/ocf1.0/download/ocf10.htm, section 4. +# https://www.idpf.org/ocf/ocf1.0/download/ocf10.htm, section 4. # From: Ralf Brown <ralf.brown@gmail.com> >>50 string epub+zip EPUB document !:mime application/epub+zip @@ -1306,6 +1357,16 @@ >5 byte x \b.%d >6 belong x (%d bytes) +# From: Joerg Jenderek +# URL: https://help.foxitsoftware.com/kb/install-fzip-file.php +# reference: http://mark0.net/download/triddefs_xml.7z/ +# defs/f/fzip.trid.xml +# Note: unknown compression; No "PK" zip magic; normally in directory like +# "%APPDATA%\Foxit Software\Addon\Foxit Reader\Install" +0 ubequad 0x2506781901010000 Foxit add-on/update +!:mime application/x-fzip +!:ext fzip + # From: "Robert Dale" <robdale@gmail.com> 0 belong 123 dar archive, >4 belong x label "%.8x @@ -1316,7 +1377,7 @@ >14 beshort 0x4e53 multi-part, with -S # Symbian installation files -# http://www.thouky.co.uk/software/psifs/sis.html +# https://www.thouky.co.uk/software/psifs/sis.html # http://developer.symbian.com/main/downloads/papers/SymbianOSv91/softwareinstallsis.pdf 8 lelong 0x10000419 Symbian installation file !:mime application/vnd.symbian.install @@ -1334,24 +1395,74 @@ >10 string x with compression level %.1s # xar (eXtensible ARchiver) archive -# xar archive format: http://code.google.com/p/xar/ +# URL: https://en.wikipedia.org/wiki/Xar_(archiver) +# xar archive format: https://code.google.com/p/xar/ # From: "David Remahl" <dremahl@apple.com> +# Update: Joerg Jenderek +# TODO: lzma compression; X509Data for pkg and xip +# Note: verified by `xar --dump-header -f FullBundleUpdate.xar` or +# 7z t -txar Xcode_10.2_beta_4.xip` 0 string xar! xar archive !:mime application/x-xar -#>4 beshort x header size %d ->6 beshort x version %d, -#>8 quad x compressed TOC: %d, -#>16 quad x uncompressed TOC: %d, +# pkg for Mac OSX installer package like FullBundleUpdate.pkg +# xip for signed Apple software like Xcode_10.2_beta_4.xip +!:ext xar/pkg/xip +# always 28 in older archives +>4 ubeshort >28 \b, header size %u +# currently there exit only version 1 since about 2014 +>6 ubeshort >1 version %u, +>8 ubequad x compressed TOC: %llu, +#>16 ubequad x uncompressed TOC: %llu, +# cksum_alg 0-2 in older and also 3-4 in newer >24 belong 0 no checksum >24 belong 1 SHA-1 checksum >24 belong 2 MD5 checksum +>24 belong 3 SHA-256 checksum +>24 belong 4 SHA-512 checksum +>24 belong >4 unknown 0x%x checksum +#>24 belong >4 checksum +# For no compression jump 0 bytes +>24 belong 0 +>>0 ubyte x +# jump more bytes forward by header size +>>>&(4.S) ubyte x +# jump more bytes forward by compressed table of contents size +#>>>>&(8.Q) ubequad x \b, heap data 0x%llx +>>>>&(8.Q) ubyte x +# look for data by ./compress after message with 1 space at end +>>>>>&-3 indirect x \b, contains +# For SHA-1 jump 20 minus 2 bytes +>24 belong 1 +>>18 ubyte x +# jump more bytes forward by header size +>>>&(4.S) ubyte x +# jump more bytes forward by compressed table of contents size +>>>>&(8.Q) ubyte x +# data compressed by gzip, bzip, lzma or none +>>>>>&-1 indirect x \b, contains +# For SHA-256 jump 32 minus 2 bytes +>24 belong 3 +>>30 ubyte x +# jump more bytes forward by header size +>>>&(4.S) ubyte x +# jump more bytes forward by compressed table of contents size +>>>>&(8.Q) ubyte x +>>>>>&-1 indirect x \b, contains +# For SHA-512 jump 64 minus 2 bytes +>24 belong 4 +>>62 ubyte x +# jump more bytes forward by header size +>>>&(4.S) ubyte x +# jump more bytes forward by compressed table of contents size +>>>>&(8.Q) ubyte x +>>>>>&-1 indirect x \b, contains # Type: Parity Archive # From: Daniel van Eeden <daniel_e@dds.nl> 0 string PAR2 Parity Archive Volume Set # Bacula volume format. (Volumes always start with a block header.) -# URL: http://bacula.org/3.0.x-manuals/en/developers/developers/Block_Header.html +# URL: https://bacula.org/3.0.x-manuals/en/developers/developers/Block_Header.html # From: Adam Buchbinder <adam.buchbinder@gmail.com> 12 string BB02 Bacula volume >20 bedate x \b, started %s @@ -1366,11 +1477,11 @@ 0 string zPQ ZPAQ stream >3 byte x \b, level %d # From: Barry Carter <carter.barry@gmail.com> -# http://encode.ru/threads/456-zpaq-updates/page32 +# https://encode.ru/threads/456-zpaq-updates/page32 0 string 7kSt ZPAQ file # BBeB ebook, unencrypted (LRF format) -# URL: http://www.sven.de/librie/Librie/LrfFormat +# URL: https://www.sven.de/librie/Librie/LrfFormat # From: Adam Buchbinder <adam.buchbinder@gmail.com> 0 string L\0R\0F\0\0\0 BBeB ebook data, unencrypted >8 beshort x \b, version %d @@ -1380,8 +1491,8 @@ >44 beshort x %d) # Symantec GHOST image by Joerg Jenderek at May 2014 -# http://us.norton.com/ghost/ -# http://www.garykessler.net/library/file_sigs.html +# https://us.norton.com/ghost/ +# https://www.garykessler.net/library/file_sigs.html 0 ubelong&0xFFFFf7f0 0xFEEF0100 Norton GHost image # *.GHO >2 ubyte&0x08 0x00 \b, first file @@ -1427,3 +1538,55 @@ # LyNX archive 56 string USE\040LYNX\040TO\040DISSOLVE\040THIS\040FILE LyNX archive + +# From: Joerg Jenderek +# URL: https://www.acronis.com/ +# Reference: https://en.wikipedia.org/wiki/TIB_(file_format) +# Note: only tested with True Image 2013 Build 5962 and 2019 Build 14110 +0 ubequad 0xce24b9a220000000 Acronis True Image backup +!:mime application/x-acronis-tib +!:ext tib +# 01000000 +#>20 ubelong x \b, at 20 0x%x +# 20000000 +#>28 ubelong x \b, at 28 0x%x +# strings like "Generic- SD/MMC 1.00" "Unknown Disk" "Msft Virtual Disk 1.0" +# ??? +# strings like "\Device\0000011e" "\Device\0000015a" +#>0 search/0x6852300/cs \\Device\\ +#>>&-1 pstring x \b, %s +# "\Device\HarddiskVolume30" "\Device\HarddiskVolume39" +#>>>&1 search/180/cs \\Device\\ +#>>>>&-1 pstring x \b, %s +#>>>>>&0 search/29/cs \0\0\xc8\0 +# disk label +#>>>>>>&10 lestring16 x \b, disk label %11.11s +#>>>>>>&9 plestring16 x \b, disk label "%11.11s" +#>>>>>>&10 ubequad x %16.16llx + + +# Gentoo XPAK binary package +# by Michal Gorny <mgorny@gentoo.org> +# https://gitweb.gentoo.org/proj/portage.git/tree/man/xpak.5 +-4 string STOP +>-16 string XPAKSTOP Gentoo binary package (XPAK) + +# From: Joerg Jenderek +# URL: https://kodi.wiki/view/TexturePacker +# Reference: https://mirrors.kodi.tv/releases/source/17.3-Krypton.tar.gz +# /xbmc-Krypton/xbmc/guilib/XBTF.h +# /xbmc-Krypton/xbmc/guilib/XBTF.cpp +0 string XBTF +# skip ASCII text by looking for terminating \0 of path +>264 ubyte 0 XBMC texture package +!:mime application/x-xbmc-xbt +!:ext xbt +# XBTF_VERSION 2 +>>4 string !2 \b, version %-.1s +# nofFiles /xbmc-Krypton/xbmc/guilib/XBTFReader.cpp +>>5 ulelong x \b, %u file +# plural s +>>5 ulelong >1 \bs +# path[CXBTFFile[MaximumPathLength=256] +>>9 string x \b, 1st %s + diff --git a/magic/Magdir/audio b/magic/Magdir/audio index 1c92cf5d9fae..5492635dfc1d 100644 --- a/magic/Magdir/audio +++ b/magic/Magdir/audio @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: audio,v 1.89 2018/07/03 20:55:37 christos Exp $ +# $File: audio,v 1.111 2019/05/08 18:02:45 christos Exp $ # audio: file(1) magic for sound formats (see also "iff") # # Jan Nicolai Langfeldt (janl@ifi.uio.no), Dan Quinlan (quinlan@yggdrasil.com), @@ -393,9 +393,15 @@ 0 string MED MED_Song 0 string SymM Symphonie SymMOD music file # +# Track Length (TRL), Tracks (TRK), Samples (SMP), Subsongs (SS) +# http://lclevy.free.fr/exotica/ahx/ahxformat.txt 0 string THX AHX version >3 byte =0 1 module data >3 byte =1 2 module data +>10 byte x TRL: %u +>11 byte x TRK: %u +>12 byte x SMP: %u +>13 byte x SS: %u # 0 string OKTASONG Oktalyzer module data # @@ -535,6 +541,18 @@ # adlib sound files # From: Alex Myczko <alex@aiei.ch> + +# https://github.com/rerrahkr/BambooTracker +0 string BambooTrackerMod BambooTracker module +>22 byte x \b, version %u +>21 byte x \b.%u +>20 byte x \b.%u + +0 string BambooTrackerIst BambooTracker instrument +>22 byte x \b, version %u +>21 byte x \b.%u +>20 byte x \b.%u + 0 string RAWADATA RdosPlay RAW 1068 string RoR AMUSIC Adlib Tracker @@ -550,6 +568,14 @@ 0 string ofTAZ! eXtra Simple Music +0 string FMK! FM Kingtracker Song + +0 string DFM DFM Song + +0 string \<CUD-FM-File\> CFF Song + +0 string _A2module A2M Song + # Spectrum 128 tunes (.ay files). # From: Emanuel Haupt <ehaupt@critical.ch> 0 string ZXAYEMUL Spectrum 128 tune @@ -631,7 +657,7 @@ # IMY # from http://filext.com/detaillist.php?extdetail=IMY -# http://cellphones.about.com/od/cellularfaqs/f/rf_imelody.htm +# https://cellphones.about.com/od/cellularfaqs/f/rf_imelody.htm # http://download.ncl.ie/doc/api/ie/ncl/media/music/IMelody.html # http://www.wx800.com/msg/download/irda/iMelody.pdf 0 string BEGIN:IMELODY iMelody Ringtone Format @@ -647,6 +673,8 @@ # URL: http://filext.com/detaillist.php?extdetail=AMR # From: Russell Coker <russell@coker.com.au> 0 string #!AMR Adaptive Multi-Rate Codec (GSM telephony) +!:mime audio/amr +!:ext amr # Type: SuperCollider 3 Synth Definition File Format # From: Mario Lang <mlang@debian.org> @@ -654,12 +682,12 @@ >4 belong x version %d # Type: True Audio Lossless Audio -# URL: http://wiki.multimedia.cx/index.php?title=True_Audio +# URL: https://wiki.multimedia.cx/index.php?title=True_Audio # From: Mike Melanson <mike@multimedia.cx> 0 string TTA1 True Audio Lossless Audio # Type: WavPack Lossless Audio -# URL: http://wiki.multimedia.cx/index.php?title=WavPack +# URL: https://wiki.multimedia.cx/index.php?title=WavPack # From: Mike Melanson <mike@multimedia.cx> 0 string wvpk WavPack Lossless Audio @@ -667,6 +695,8 @@ # VGM music file 0 string Vgm\040 >9 ubyte >0 VGM Video Game Music dump v +!:mime audio/x-vgm +!:ext vgm >>9 ubyte/16 >0 \b%d >>9 ubyte&0x0F x \b%d >>8 ubyte/16 x \b.%d @@ -713,6 +743,54 @@ >>>>0x78 ubyte 0x03 AY-3-8930, >>>>0x78 ubyte 0x10 YM2149, >>>>0x78 ubyte 0x11 YM3439, +# VGM 1.61 +>>0x34 ulelong >0x4C +>>>0x80 ulelong >0 DMG, +>>0x34 ulelong >0x50 +>>>0x84 lelong >0 NES APU, +>>>0x84 lelong <0 NES APU with FDS, +>>0x34 ulelong >0x54 +>>>0x88 ulelong >0 MultiPCM, +>>0x34 ulelong >0x58 +>>>0x8C ulelong >0 uPD7759, +>>0x34 ulelong >0x5C +>>>0x90 ulelong >0 OKIM6258, +>>0x34 ulelong >0x64 +>>>0x98 ulelong >0 OKIM6295, +>>0x34 ulelong >0x68 +>>>0x9C ulelong >0 K051649, +>>0x34 ulelong >0x6C +>>>0xA0 ulelong >0 K054539, +>>0x34 ulelong >0x70 +>>>0xA4 ulelong >0 HuC6280, +>>0x34 ulelong >0x74 +>>>0xA8 ulelong >0 C140, +>>0x34 ulelong >0x78 +>>>0xAC ulelong >0 K053260, +>>0x34 ulelong >0x7C +>>>0xB0 ulelong >0 Pokey, +>>0x34 ulelong >0x80 +>>>0xB4 ulelong >0 QSound, +# VGM 1.71 +>>0x34 ulelong >0x84 +>>>0xB8 ulelong >0 SCSP, +>>0x34 ulelong >0x8C +>>>0xC0 ulelong >0 WonderSwan, +>>0x34 ulelong >0x90 +>>>0xC4 ulelong >0 VSU, +>>0x34 ulelong >0x94 +>>>0xC8 ulelong >0 SAA1099, +>>0x34 ulelong >0x98 +>>>0xCC ulelong >0 ES5503, +>>0x34 ulelong >0x9C +>>>0xD0 lelong >0 ES5505, +>>>0xD0 lelong <0 ES5506, +>>0x34 ulelong >0xA4 +>>>0xD8 ulelong >0 X1-010, +>>0x34 ulelong >0xA8 +>>>0xDC ulelong >0 C352, +>>0x34 ulelong >0xAC +>>>0xE0 ulelong >0 GA20, # GVOX Encore file format # Since this is a proprietary file format and there is no publicly available @@ -727,8 +805,8 @@ # Summary: Garmin Voice Processing Module (WAVE audios) # From: Joerg Jenderek -# URL: http://www.garmin.com/ -# Reference: http://turboccc.wikispaces.com/share/view/28622555 +# URL: https://www.garmin.com/ +# Reference: http://www.poi-factory.com/node/19580 # NOTE: there exist 2 other Garmin VPM formats 0 string AUDIMG # skip text files starting with string "AUDIMG" @@ -752,16 +830,35 @@ # if you select a language like german on your garmin device # you can only select voice modules with corresponding language byte ID like 1 >>18 ubyte x \b, language ID %d +# structure for phrases/sentences? +# number of voice sample in the 1st phrase? +#>>19 uleshort x \b, 0x%x samples +#>>>21 uleshort >0 \b, at 0x%4.4x +#>>>(21.s) ubequad x 0x%llx +# 2nd phrase? +#>>23 uleshort x \b, 0x%x samples +#>>>25 uleshort >0 \b, at 0x%4.4x +#>>>(25.s) ubequad x 0x%llx # pointer to 1st audio WAV sample >>16 uleshort >0 ->>>(16.s) ulelong >0 \b, at offset 0x%x +>>>(16.s) ulelong >0 \b, at 0x%x # WAV length ->>>>(16.s+4) ulelong >0 %d Bytes +# 1 space char after "bytes" to get phrase "bytes RIFF" +>>>>(16.s+4) ulelong >0 %u bytes # look for magic >>>>>(&-8.l) string RIFF # determine type by ./riff ->>>>>>&-4 indirect x \b +>>>>>>&-4 indirect x # 2 - ~ 131 WAV samples following same way +# +# Summary: encrypted Garmin Voice Processing Module +# From: Joerg Jenderek +# URL: https://www.garmin.com/us/products/ontheroad/voicestudio +# NOTE: Encrypted variant used in voices like DrNightmare, Elfred, Yeti. +# There exist 2 other Garmin VPM formats +0 ubequad 0xa141190fecc8ced6 Garmin Voice Processing Module (encrypted) +!:mime audio/x-vpm-garmin +!:ext vpm # From Martin Mueller Skarbiniks Pedersen 0 string GDM @@ -816,7 +913,7 @@ >>>0x31 byte&0x0F x \b%02d >>>>0x4 string >\0 \b, title: "%s" -# magic for Klystrack, http://kometbomb.github.io/klystrack/ +# magic for Klystrack, https://kometbomb.github.io/klystrack/ # from Alex Myczko <alex@aiei.ch> 0 string cyd!song Klystrack song >8 byte >0 \b, version %u @@ -866,11 +963,15 @@ # Added by David Korth <gerbilsoft@gerbilsoft.com> 0x00 beshort 0x8000 >(2.S-2) string (c)CRI CRI ADX ADPCM audio +!:ext adx +!:mime audio/x-adx +!:strength +50 >>0x12 byte x v%u >>0x04 byte 0x02 \b, pre-set prediction coefficients >>0x04 byte 0x03 \b, standard ADX >>0x04 byte 0x04 \b, exponential scale ->>0x04 byte 0x05 \b, AHX +>>0x04 byte 0x10 \b, AHX (Dreamcast) +>>0x04 byte 0x11 \b, AHX >>0x08 belong x \b, %u Hz >>0x12 byte 0x03 >>>0x02 beshort >0x2B @@ -888,3 +989,125 @@ # Sony PlayStation Audio (.xa) 0 leshort 0x4158 Sony PlayStation Audio +# Portable Sound Format +# Used for audio rips for various consoles. +# http://fileformats.archiveteam.org/wiki/Portable_Sound_Format +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string PSF Portable Sound Format +!:mime audio/x-psf +>3 byte 0x01 (Sony PlayStation) +>3 byte 0x02 (Sony PlayStation 2) +>3 byte 0x11 (Sega Saturn) +>3 byte 0x12 (Sega Dreamcast) +>3 byte 0x13 (Sega Mega Drive) +>3 byte 0x21 (Nintendo 64) +>3 byte 0x22 (Game Boy Advance) +>3 byte 0x23 (Super NES) +>3 byte 0x41 (Capcom QSound) + +# Atari 8-bit SAP audio format +# http://asap.sourceforge.net/sap-format.html +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string SAP\r\n Atari 8-bit SAP audio file +!:mime audio/x-sap +!:ext sap +>5 search/1024 NAME +>>&1 string x \b: %s +>>5 search/1024 AUTHOR +>>>&1 string x by %s + +# Nintendo Wii BRSTM audio format (fields) +# NOTE: Assuming HEAD starts at 0x40. +# FIXME: Replace 0x48 with HEAD offset plus 8. +0 name nintendo-wii-brstm-fields +>(0x10.L) string HEAD \b: +>>(0x10.L+0x0C) belong x +>>>(&-4.L+0x48) belong x +>>>>&-4 byte 0 PCM, signed 8-bit, +>>>>&-4 byte 1 PCM, signed 16-bit, +>>>>&-4 byte 2 THP ADPCM, +>>>>&-3 byte !0 looping, +>>>>&-2 byte 1 mono +>>>>&-2 byte 2 stereo +>>>>&-2 byte 3 3 channels +>>>>&-2 byte 4 quad +>>>>&-2 byte >4 %u channels +>>>>&0 beshort !0 %u Hz + +# Nintendo Wii BRSTM audio format +# https://wiibrew.org/wiki/BRSTM_file +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string RSTM Nintendo Wii BRSTM audio file +!:mime audio/x-brstm +!:ext brstm +# Wii is big-endian, so default to BE. +>4 beshort 0xFEFF +>>0 use nintendo-wii-brstm-fields +>4 leshort 0xFEFF +>>0 use \^nintendo-wii-brstm-fields + +# Nintendo 3DS BCSTM audio format (fields) +0 name nintendo-3ds-bcstm-fields +>(0x18.l) string INFO \b: +# INFO block: Stream information starts at 0x20 (minus 4 for the 'INFO' magic) +>>&0x1C byte 0 PCM, signed 8-bit, +>>&0x1C byte 1 PCM, signed 16-bit, +>>&0x1C byte 2 DSP ADPCM, +>>&0x1C byte 3 IMA ADPCM, +>>&0x1D byte !0 looping, +>>&0x1E byte 1 mono +>>&0x1E byte 2 stereo +>>&0x1E byte 3 3 channels +>>&0x1E byte 4 quad +>>&0x1E byte >4 %u channels +>>&0x20 lelong !0 %u Hz + +# Nintendo 3DS BCSTM audio format +# https://www.3dbrew.org/wiki/BCSTM +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string CSTM Nintendo 3DS BCSTM audio file +!:mime audio/x-bcstm +!:ext bcstm +# 3DS is little-endian, so default to LE. +>4 leshort 0xFEFF +>>0 use nintendo-3ds-bcstm-fields +>4 beshort 0xFEFF +>>0 use \^nintendo-3ds-bcstm-fields + +# Nintendo Wii U BFSTM audio format +# http://mk8.tockdom.com/wiki/BFSTM_(File_Format) +# NOTE: This format is very similar to BCSTM. +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string FSTM Nintendo Wii U BFSTM audio file +!:mime audio/x-bfstm +!:ext bfstm +# BFSTM is used on both Wii U (BE) and Switch (LE), +# so default to LE. +>4 leshort 0xFEFF +>>0 use nintendo-3ds-bcstm-fields +>4 beshort 0xFEFF +>>0 use \^nintendo-3ds-bcstm-fields + +# Nintendo 3DS BCSTM audio format (fields) +0 name nintendo-3ds-bcwav-fields +>(0x18.l) string INFO \b: +# INFO block (minus 4 for INFO magic) +>>&0x4 byte 0 PCM, signed 8-bit, +>>&0x4 byte 1 PCM, signed 16-bit, +>>&0x4 byte 2 DSP ADPCM, +>>&0x4 byte 3 IMA ADPCM, +>>&0x5 byte !0 looping, +>>&0x8 lelong x stereo +>>&0x8 lelong !0 %u Hz + +# Nintendo 3DS BCWAV audio format +# https://www.3dbrew.org/wiki/BCWAV +# Added by David Korth <gerbilsoft@gerbilsoft.com> +0 string CWAV Nintendo 3DS BCWAV audio file +!:mime audio/x-bcwav +!:ext bcwav +# 3DS is little-endian, so default to LE. +>4 leshort 0xFEFF +>>0 use nintendo-3ds-bcwav-fields +>4 beshort 0xFEFF +>>0 use \^nintendo-3ds-bcwav-fields diff --git a/magic/Magdir/basis b/magic/Magdir/basis index 128aa4c77c2d..19dd463b414b 100644 --- a/magic/Magdir/basis +++ b/magic/Magdir/basis @@ -1,9 +1,9 @@ #---------------------------------------------------------------- -# $File: basis,v 1.4 2009/09/19 16:28:08 christos Exp $ +# $File: basis,v 1.5 2019/04/19 00:42:27 christos Exp $ # basis: file(1) magic for BBx/Pro5-files # Oliver Dammer <dammer@olida.de> 2005/11/07 -# http://www.basis.com business-basic-files. +# https://www.basis.com business-basic-files. # 0 string \074\074bbx\076\076 BBx >7 string \000 indexed file diff --git a/magic/Magdir/ber b/magic/Magdir/ber index db7a8613d91a..15288c682416 100644 --- a/magic/Magdir/ber +++ b/magic/Magdir/ber @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: ber,v 1.1 2016/06/05 00:21:30 christos Exp $ +# $File: ber,v 1.2 2019/04/19 00:42:27 christos Exp $ # ber: file(1) magic for several BER formats used in the mobile # telecommunications industry (Georg Sauthoff) @@ -19,7 +19,7 @@ # # TAP 3 Files # TAP -> Transferred Account Procedure -# cf. http://www.gsma.com/newsroom/wp-content/uploads/TD.57-v32.31.pdf +# cf. https://www.gsma.com/newsroom/wp-content/uploads/TD.57-v32.31.pdf # TransferBatch short tag 0 byte 0x61 # BatchControlInfo short tag @@ -47,7 +47,7 @@ >>&0 byte x NRT 2.%d (TD.35, Near Real Time Roaming Data Exchange) # RAP Files -# cf. http://www.gsma.com/newsroom/wp-content/uploads/TD.32-v6.11.pdf +# cf. https://www.gsma.com/newsroom/wp-content/uploads/TD.32-v6.11.pdf # Long ReturnBatch tag 0 string \x7f\x84\x16 # Long RapBatchControlInfo tag diff --git a/magic/Magdir/bioinformatics b/magic/Magdir/bioinformatics index 0c761ed351fe..2966fa6e49d1 100644 --- a/magic/Magdir/bioinformatics +++ b/magic/Magdir/bioinformatics @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: bioinformatics,v 1.4 2016/06/20 16:13:46 christos Exp $ +# $File: bioinformatics,v 1.5 2019/04/19 00:42:27 christos Exp $ # bioinfomatics: file(1) magic for Bioinfomatics file formats ############################################################################### @@ -83,7 +83,7 @@ ############################################################################### # BCF (Binary Call Format), version 2.1 -# used by SAMtools (http://samtools.github.io/hts-specs/BCFv2_qref.pdf) +# used by SAMtools (https://samtools.github.io/hts-specs/BCFv2_qref.pdf) # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it ############################################################################### 0 string BCF\2\1 Binary Call Format (BCF) version 2.1 @@ -96,7 +96,7 @@ ############################################################################### # BCF (Binary Call Format), version 2.2 -# used by SAMtools (http://samtools.github.io/hts-specs/BCFv2_qref.pdf) +# used by SAMtools (https://samtools.github.io/hts-specs/BCFv2_qref.pdf) # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it ############################################################################### 0 string BCF\2\2 Binary Call Format (BCF) version 2.2 @@ -129,7 +129,7 @@ ############################################################################### # FASTA -# used by FASTA (http://fasta.bioch.virginia.edu/fasta_www2/fasta_guide.pdf) +# used by FASTA (https://fasta.bioch.virginia.edu/fasta_www2/fasta_guide.pdf) ############################################################################### #0 byte 0x3e # q>0 regex =^[>][!-~\t\ ]+$ diff --git a/magic/Magdir/biosig b/magic/Magdir/biosig new file mode 100644 index 000000000000..e490f6cc7eff --- /dev/null +++ b/magic/Magdir/biosig @@ -0,0 +1,154 @@ + +############################################################################## +# +# Magic ids for biomedical signal file formats +# Copyright (C) 2018 Alois Schloegl <alois.schloegl@gmail.com> +# +# The list has been derived from biosig projects +# http://biosig.sourceforge.net +# https://pub.ist.ac.at/~schloegl/matlab/eeg/ +# https://pub.ist.ac.at/~schloegl/biosig/TESTED +# +############################################################################## +# +0 string ABF\x20 Biosig/Axon Binary format +!:mime biosig/abf2 +0 string ABF2\0\0 Biosig/Axon Binary format +!:mime biosig/abf2 +# +0 string ATES\x20MEDICA\x20SOFT.\x20EEG\x20for\x20Windows Biosig/ATES MEDICA SOFT. EEG for Windows +!:mime biosig/ates +# +0 string ATF\x09 Biosig/Axon Text fomrat +!:mime biosig/atf +# +0 string ADU1 Biosig/Axona file format +!:mime biosig/axona +0 string ADU2 Biosig/Axona file format +!:mime biosig/axona +# +0 string ALPHA-TRACE-MEDICAL Biosig/alpha trace +!:mime biosig/alpha +# +0 string AxGr Biosig/AXG +0 string axgx Biosig/AXG +!:mime biosig/axg +# +0 string HeaderLen= Biosig/BCI2000 +0 string BCI2000V Biosig/BCI2000 +!:mime biosig/bci2000 +# +### Specification: https://www.biosemi.com/faq/file_format.htm +0 string \xffBIOSEMI Biosig/Biosemi data format +!:mime biosig/bdf +# +0 string Brain\x20Vision\x20Data\x20Exchange\x20Header\x20File Biosig/Brainvision data file +0 string Brain\x20Vision\x20V-Amp\x20Data\x20Header\x20File\x20Version Biosig/Brainvision V-Amp file +0 string Brain\x20Vision\x20Data\x20Exchange\x20Marker\x20File,\x20Version Biosig/Brainvision Marker file +!:mime biosig/brainvision +# +0 string CEDFILE Biosig/CFS: Cambridge Electronic devices File format +!:mime biosig/ced +# +### Specification: https://www.edfplus.info/specs/index.html +0 string 0\x20\x20\x20\x20\x20\x20\x20 Biosig/EDF: European Data format +!:mime biosig/edf +# +### Specifications: https://arxiv.org/abs/cs/0608052 +0 string GDF Biosig/GDF: General data format for biosignals +!:mime biosig/gdf +# +0 string DATA\0\0\0\0 Biosig/Heka Patchmaster +0 string DAT1\0\0\0\0 Biosig/Heka Patchmaster +0 string DAT2\0\0\0\0 Biosig/Heka Patchmaster +!:mime biosig/heka +# +0 string (C)\x20CED\x2087 Biosig/CED SMR +!:mime biosig/ced-smr +# +0 string CFWB\1\0\0\0 Biosig/CFWB +!:mime biosig/cfwb +# +0 string DEMG Biosig/DEMG +!:mime biosig/demg +# +0 string EBS\x94\x0a\x13\x1a\x0d Biosig/EBS +!:mime biosig/ebs +# +0 string Embla\x20data\x20file Biosig/Embla +!:mime biosig/embla +# +0 string Header\r\nFile Version Biosig/ETG4000 +!:mime biosig/etg4000 +# +0 string GALILEO\x20EEG\x20TRACE\x20FILE Biosig/Galileo +!:mime biosig/galileo +# +0 string IGOR Biosig/IgorPro ITX file +!:mime biosig/igorpro +# +# Specification: http://www.ampsmedical.com/uploads/2017-12-7/The_ISHNE_Format.pdf +0 string ISHNE1.0 Biosig/ISHNE +!:mime biosig/ishne +# +# CEN/ISO 11073/22077 series, http://www.mfer.org/en/document.htm +0 string @\x20\x20MFER\x20 Biosig/MFER +0 string @\x20MFR\x20 Biosig/MFER +!:mime biosig/mfer +# +0 string NEURALEV Biosig/NEV +0 string N.EV.\0 Biosig/NEV +!:mime biosig/nev +# +0 string NEX1 Biosig/NEX +!:mime biosig/nex1 +# +0 string PLEX Biosig/Plexon v1.0 +10 string PLEXON Biosig/Plexon v2.0 +!:mime biosig/plexon +# +0 string \x02\x27\x91\xC6 Biosig/RHD2000: Intan RHD2000 format +# +# Specification: CEN 1064:2005/ISO 11073:91064 +16 string SCPECG\0\0 Biosig/SCP-ECG format CEN 1064:2005/ISO 11073:91064 +!:mime biosig/scpecg +# +0 string IAvSFo Biosig/SIGIF +!:mime biosig/sigif +# +0 string POLY\x20SAMPLE\x20FILEversion\x20 Biosig/TMS32 +!:mime biosig/tms32 +# +0 string FileId=TMSi\x20PortiLab\x20sample\x20log\x20file\x0a\x0dVersion= Biosig/TMSiLOG +!:mime biosig/tmsilog +# +4 string Synergy\0\48\49\50\46\48\48\51\46\48\48\48\46\48\48\48\0\28\0\0\0\2\0\0\0 +>63 string CRawDataElement +>>85 string CRawDataBuffer Biosig/SYNERGY +!:mime biosig/synergy +# +4 string \40\0\4\1\44\1\102\2\146\3\44\0\190\3 Biosig/UNIPRO +!:mime biosig/unipro +# +0 string VER=9\r\nCTIME= Biosig/WCP +!:mime biosig/wcp +# +0 string \xAF\xFE\xDA\xDA Biosig/Walter Graphtek +0 string \xDA\xDA\xFE\xAF Biosig/Walter Graphtek +0 string \x55\x55\xFE\xAF Biosig/Walter Graphtek +!:mime biosig/walter-graphtek +# +0 string V3.0\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20 +>32 string [PatInfo] Biosig/Sigma +!:mime biosig/sigma +# +0 string \067\069\078\013\010\0x1a\04\0x84 Biosig/File exchange format (FEF) +!:mime biosig/fef +0 string \67\69\78\0x13\0x10\0x1a\4\0x84 Biosig/File exchange format (FEF) +!:mime biosig/fef +# +0 string \0\0\0\x64\0\0\0\x1f\0\0\0\x14\0\0\0\0\0\1 +>36 string \0\0\0\x65\0\0\0\3\0\0\0\4\0\0 +>>56 string \0\0\0\x6a\0\0\0\3\0\0\0\4\0\0\0\0\xff\xff\xff\xff\0\0 Biosig/FIFF +!:mime biosig/fiff +# diff --git a/magic/Magdir/blcr b/magic/Magdir/blcr index 70ea563e182d..d2f901ae921e 100644 --- a/magic/Magdir/blcr +++ b/magic/Magdir/blcr @@ -1,5 +1,5 @@ # Berkeley Lab Checkpoint Restart (BLCR) checkpoint context files -# http://ftg.lbl.gov/checkpoint +# https://ftg.lbl.gov/checkpoint 0 string C\0\0\0R\0\0\0 BLCR >16 lelong 1 x86 >16 lelong 3 alpha diff --git a/magic/Magdir/blender b/magic/Magdir/blender index 09485702b15b..276242eab02f 100644 --- a/magic/Magdir/blender +++ b/magic/Magdir/blender @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: blender,v 1.7 2017/03/17 21:35:28 christos Exp $ +# $File: blender,v 1.8 2019/04/19 00:42:27 christos Exp $ # blender: file(1) magic for Blender 3D related files # # Native format rule v1.2. For questions use the developers list -# http://lists.blender.org/mailman/listinfo/bf-committers +# https://lists.blender.org/mailman/listinfo/bf-committers # GLOB chunk was moved near start and provides subversion info since 2.42 0 string =BLENDER Blender3D, diff --git a/magic/Magdir/c-lang b/magic/Magdir/c-lang index 7b3f703ef3ec..becf6b02ecca 100644 --- a/magic/Magdir/c-lang +++ b/magic/Magdir/c-lang @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: c-lang,v 1.26 2017/08/14 07:40:38 christos Exp $ +# $File: c-lang,v 1.27 2019/02/27 16:46:23 christos Exp $ # c-lang: file(1) magic for C and related languages programs # # The strength is to beat standard HTML @@ -12,31 +12,41 @@ # C # Check for class if include is found, otherwise class is beaten by include becouse of lowered strength -0 regex \^#include C ->0 regex \^class[[:space:]]+ ->>&0 regex \\{[\.\*]\\}(;)?$ \b++ ->&0 clear x source text +0 search/8192 #include +>0 regex \^#include C +>>0 regex \^class[[:space:]]+ +>>>&0 regex \\{[\.\*]\\}(;)?$ \b++ +>>&0 clear x source text !:strength + 13 !:mime text/x-c -0 regex \^#[[:space:]]*pragma C source text +0 search/8192 pragma +>0 regex \^#[[:space:]]*pragma C source text !:mime text/x-c -0 regex \^#[[:space:]]*(if\|ifn)def ->&0 regex \^#[[:space:]]*endif$ C source text +0 search/8192 endif +>0 regex \^#[[:space:]]*(if\|ifn)def +>>&0 regex \^#[[:space:]]*endif$ C source text !:mime text/x-c -0 regex \^#[[:space:]]*(if\|ifn)def ->&0 regex \^#[[:space:]]*define C source text +0 search/8192 define +>0 regex \^#[[:space:]]*(if\|ifn)def +>>&0 regex \^#[[:space:]]*define C source text !:mime text/x-c -0 regex \^[[:space:]]*char(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text +0 search/8192 char +>0 regex \^[[:space:]]*char(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text !:mime text/x-c -0 regex \^[[:space:]]*double(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text +0 search/8192 double +>0 regex \^[[:space:]]*double(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text !:mime text/x-c -0 regex \^[[:space:]]*extern[[:space:]]+ C source text +0 search/8192 extern +>0 regex \^[[:space:]]*extern[[:space:]]+ C source text !:mime text/x-c -0 regex \^[[:space:]]*float(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text +0 search/8192 float +>0 regex \^[[:space:]]*float(\ \\*|\\*)(.+)(=.*)?;[[:space:]]*$ C source text !:mime text/x-c -0 regex \^struct[[:space:]]+ C source text +0 search/8192 struct +>0 regex \^struct[[:space:]]+ C source text !:mime text/x-c -0 regex \^union[[:space:]]+ C source text +0 search/8192 union +>0 regex \^union[[:space:]]+ C source text !:mime text/x-c 0 search/8192 main( >&0 regex \\)[[:space:]]*\\{ C source text @@ -44,35 +54,44 @@ # C++ # The strength of these rules is increased so they beat the C rules above -0 regex \^namespace[[:space:]]+[_[:alpha:]]{1,30}[[:space:]]*\\{ C++ source text +0 search/8192 namespace +>0 regex \^namespace[[:space:]]+[_[:alpha:]]{1,30}[[:space:]]*\\{ C++ source text !:strength + 30 !:mime text/x-c++ # using namespace [namespace] or using std::[lib] -0 regex \^using[[:space:]]+(namespace\ )?std(::)?[[:alpha:]]*[[:space:]]*; C++ source text +0 search/8192 using +>0 regex \^using[[:space:]]+(namespace\ )?std(::)?[[:alpha:]]*[[:space:]]*; C++ source text !:strength + 30 !:mime text/x-c++ -0 regex \^[[:space:]]*template[[:space:]]*<.*>[[:space:]]*$ C++ source text +0 search/8192 template +>0 regex \^[[:space:]]*template[[:space:]]*<.*>[[:space:]]*$ C++ source text !:strength + 30 !:mime text/x-c++ -0 regex \^[[:space:]]*virtual[[:space:]]+.*[};][[:space:]]*$ C++ source text +0 search/8192 virtual +>0 regex \^[[:space:]]*virtual[[:space:]]+.*[};][[:space:]]*$ C++ source text !:strength + 30 !:mime text/x-c++ # But class alone is reduced to avoid beating php (Jens Schleusener) -0 regex \^[[:space:]]*class[[:space:]]+[[:digit:][:alpha:]:_]+[[:space:]]*\\{(.*[\n]*)*\\}(;)?$ C++ source text +0 search/8192 class +>0 regex \^[[:space:]]*class[[:space:]]+[[:digit:][:alpha:]:_]+[[:space:]]*\\{(.*[\n]*)*\\}(;)?$ C++ source text !:strength + 13 !:mime text/x-c++ -0 regex \^[[:space:]]*public: C++ source text +0 search/8192 public +>0 regex \^[[:space:]]*public: C++ source text !:strength + 30 !:mime text/x-c++ -0 regex \^[[:space:]]*private: C++ source text +0 search/8192 private +>0 regex \^[[:space:]]*private: C++ source text !:strength + 30 !:mime text/x-c++ -0 regex \^[[:space:]]*protected: C++ source text +0 search/8192 protected +>0 regex \^[[:space:]]*protected: C++ source text !:strength + 30 !:mime text/x-c++ # Objective-C -0 regex \^#import Objective-C source text +0 search/8192 #import +>0 regex \^#import Objective-C source text !:strength + 25 !:mime text/x-objective-c diff --git a/magic/Magdir/cad b/magic/Magdir/cad index 3a71e06b6901..48a76d14c976 100644 --- a/magic/Magdir/cad +++ b/magic/Magdir/cad @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: cad,v 1.16 2018/05/07 23:26:31 christos Exp $ +# $File: cad,v 1.19 2019/04/19 00:42:27 christos Exp $ # autocad: file(1) magic for cad files # @@ -11,12 +11,12 @@ # raster underlays to Microstation DGN (vector) drawings. # # http://www.wotsit.org/search.asp -# http://filext.com/detaillist.php?extdetail=DGN -# http://filext.com/detaillist.php?extdetail=CIT +# https://filext.com/detaillist.php?extdetail=DGN +# https://filext.com/detaillist.php?extdetail=CIT # -# http://www.bentley.com/products/default.cfm?objectid=97F351F5-9C35-4E5E-89C2 +# https://www.bentley.com/products/default.cfm?objectid=97F351F5-9C35-4E5E-89C2 # 3F86C928&method=display&p_objectid=97F351F5-9C35-4E5E-89C280A93F86C928 -# http://www.bentley.com/products/default.cfm?objectid=A5C2FD43-3AC9-4C71-B682 +# https://www.bentley.com/products/default.cfm?objectid=A5C2FD43-3AC9-4C71-B682 # 721C479F&method=display&p_objectid=A5C2FD43-3AC9-4C71-B682C7BE721C479F 0 string \010\011\376 Microstation >3 string \002 @@ -43,8 +43,8 @@ >4 string \030\000\003 CITFile # AutoCAD -# Merge of the different contributions and updates from http://en.wikipedia.org/wiki/Dwg -# and http://www.iana.org/assignments/media-types/image/vnd.dwg +# Merge of the different contributions and updates from https://en.wikipedia.org/wiki/Dwg +# and https://www.iana.org/assignments/media-types/image/vnd.dwg 0 string MC0.0 DWG AutoDesk AutoCAD Release 1.0 !:mime image/vnd.dwg 0 string AC1.2 DWG AutoDesk AutoCAD Release 1.2 @@ -89,7 +89,7 @@ # Sergey Zaykov (mail_of_sergey@mail.ru, sergey_zaikov@rambler.ru, # ICQ 358572321) # From various sources like: -# http://autodesk.blogs.com/between_the_lines/autocad-release-history.html +# https://autodesk.blogs.com/between_the_lines/autocad-release-history.html 0 string AC1018 DWG AutoDesk AutoCAD 2004/2005/2006 !:mime image/vnd.dwg 0 string AC1021 DWG AutoDesk AutoCAD 2007/2008/2009 @@ -102,13 +102,13 @@ # KOMPAS 2D drawing from ASCON # This is KOMPAS 2D drawing or fragment of drawing but is not detailed nor # gathered nor specification -# ASCON http://ascon.net/main/ in English, -# http://ascon.ru/ main site in Russian +# ASCON https://ascon.net/main/ in English, +# https://ascon.ru/ main site in Russian # Extension is CDW for drawing and FRW for fragment of drawing # Sergey Zaykov (mail_of_sergey@mail.ru, sergey_zaikov@rambler.ru, -# ICQ 358572321, http://vkontakte.ru/id16076543) +# ICQ 358572321, https://vkontakte.ru/id16076543) # From: -# http://sd.ascon.ru/otrs/customer.pl?Action=CustomerFAQ&CategoryID=4&ItemID=292 +# https://sd.ascon.ru/otrs/customer.pl?Action=CustomerFAQ&CategoryID=4&ItemID=292 # (in russian) and my experiments 0 string KF >2 belong 0x4E00000C Kompas drawing 12.0 SP1 @@ -156,7 +156,7 @@ !:ext 3ds # MegaCAD 2D/3D drawing (.prt) -# http://megacad.de/ +# https://megacad.de/ # From: Markus Heidelberg <markus.heidelberg@web.de> 0 string MegaCad23\0 MegaCAD 2D/3D drawing @@ -167,3 +167,24 @@ 0 string ;;\020HSF\020V OpenHSF (Hoops Stream Format) >7 regex/9 V[.0-9]{4,5}\020 %s !:ext hsf + +# AutoCAD Drawing Exchange Format +0 regex \^[\ \t]*0\r?\000$ +>1 regex \^[\ \t]*SECTION\r?$ +>>2 regex \^[\ \t]*2\r?$ +>>>3 regex \^[\ \t]*HEADER\r?$ AutoCAD Drawing Exchange Format +!:mime application/x-dxf +!:ext dxf +>>>>&1 search/8192 AC1006 \b, R10 +>>>>&1 search/8192 AC1009 \b, R11/R12 +>>>>&1 search/8192 AC1012 \b, R13 +>>>>&1 search/8192 AC1014 \b, R14 +>>>>&1 search/8192 AC1015 \b, version 2000 +>>>>&1 search/8192 AC1018 \b, version 2004 +>>>>&1 search/8192 AC1021 \b, version 2007 +>>>>&1 search/8192 AC1024 \b, version 2010 + +# The Sketchup 3D model format https://www.sketchup.com/ +0 string \xff\xfe\xff\x0e\x53\x00\x6b\x00\x65\x00\x74\x00\x63\x00\x68\x00\x55\x00\x70\x00\x20\x00\x4d\x00\x6f\x00\x64\x00\x65\x00\x6c\x00 SketchUp Model +!:mime application/vnd.sketchup.skp +!:ext skp diff --git a/magic/Magdir/cafebabe b/magic/Magdir/cafebabe index 6482858fc62c..18dd1a27a39f 100644 --- a/magic/Magdir/cafebabe +++ b/magic/Magdir/cafebabe @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: cafebabe,v 1.23 2017/05/25 20:07:23 christos Exp $ +# $File: cafebabe,v 1.24 2018/10/01 23:33:15 christos Exp $ # Cafe Babes unite! # # Since Java bytecode and Mach-O universal binaries have the same magic number, @@ -58,15 +58,15 @@ >>4 belong <20 Mach-O universal binary with %d architectures: !:mime application/x-mach-binary >>>8 use mach-o \b ->>4 belong 2 +>>4 belong >1 >>>28 use mach-o \b ->>4 belong 3 +>>4 belong >2 >>>48 use mach-o \b ->>4 belong 4 +>>4 belong >3 >>>68 use mach-o \b ->>4 belong 5 +>>4 belong >4 >>>88 use mach-o \b ->>4 belong 6 +>>4 belong >5 >>>108 use mach-o \b ### MACH-O END ### diff --git a/magic/Magdir/clojure b/magic/Magdir/clojure new file mode 100644 index 000000000000..1f1cddf9a2db --- /dev/null +++ b/magic/Magdir/clojure @@ -0,0 +1,30 @@ +#------------------------------------------------------------------------------ +# file: file(1) magic for Clojure +# URL: https://clojure.org/ +# From: Jason Felice <jason.m.felice@gmail.com> + +0 string/w #!\ /usr/bin/clj Clojure script text executable +!:mime text/x-clojure +0 string/w #!\ /usr/local/bin/clj Clojure script text executable +!:mime text/x-clojure +0 string/w #!\ /usr/bin/clojure Clojure script text executable +!:mime text/x-clojure +0 string/w #!\ /usr/local/bin/clojure Clojure script text executable +!:mime text/x-clojure +0 string/W #!/usr/bin/env\ clj Clojure script text executable +!:mime text/x-clojure +0 string/W #!/usr/bin/env\ clojure Clojure script text executable +!:mime text/x-clojure +0 string/W #!\ /usr/bin/env\ clj Clojure script text executable +!:mime text/x-clojure +0 string/W #!\ /usr/bin/env\ clojure Clojure script text executable +!:mime text/x-clojure + +0 regex \^\\\(ns[[:space:]]+[a-z] Clojure module source text +!:mime text/x-clojure + +0 regex \^\\\(ns[[:space:]]+\\\^\\{: Clojure module source text +!:mime text/x-clojure + +0 regex \^\\\(defn-?[[:space:]] Clojure module source text +!:mime text/x-clojure diff --git a/magic/Magdir/coff b/magic/Magdir/coff index b4addec58ffd..31b47e7aff42 100644 --- a/magic/Magdir/coff +++ b/magic/Magdir/coff @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: coff,v 1.2 2017/03/17 21:35:28 christos Exp $ +# $File: coff,v 1.3 2018/08/01 10:34:03 christos Exp $ # coff: file(1) magic for Common Object Files not specific to known cpu types or manufactures # # COFF @@ -26,6 +26,10 @@ >>0 uleshort 0x0550 Hitachi SH little-endian # executable (RISC System/6000 V3.1) or obj module (./ibm6000) #>>0 uleshort 0x01DF +# MS Windows COFF Intel Itanium, AMD64 +# https://msdn.microsoft.com/en-us/library/windows/desktop/ms680313(v=vs.85).aspx +>>0 uleshort 0x0200 Intel ia64 +>>0 uleshort 0x8664 Intel amd64 # TODO for other COFFs #>>0 uleshort 0xABCD COFF_TEMPLATE >>0 default x diff --git a/magic/Magdir/commands b/magic/Magdir/commands index f6ad1c878a31..1120c7d06f6b 100644 --- a/magic/Magdir/commands +++ b/magic/Magdir/commands @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: commands,v 1.59 2017/08/14 07:40:38 christos Exp $ +# $File: commands,v 1.60 2019/04/19 00:42:27 christos Exp $ # commands: file(1) magic for various shells and interpreters # #0 string/w : shell archive or script for antique kernel text @@ -96,7 +96,7 @@ 0 search/1/w #!\ /usr/bin/php PHP script text executable !:strength + 10 !:mime text/x-php -# Smarty compiled template, http://www.smarty.net/ +# Smarty compiled template, https://www.smarty.net/ # Elan Ruusamae <glen@delfi.ee> 0 string =<?php >5 regex [\ \n] @@ -109,7 +109,7 @@ 0 string/t $! DCL command file # Type: Pdmenu -# URL: http://packages.debian.org/pdmenu +# URL: https://packages.debian.org/pdmenu # From: Edward Betts <edward@debian.org> 0 string #!/usr/bin/pdmenu Pdmenu configuration file text diff --git a/magic/Magdir/compress b/magic/Magdir/compress index 107d317c1537..7520eb4ab0f1 100644 --- a/magic/Magdir/compress +++ b/magic/Magdir/compress @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: compress,v 1.72 2018/03/27 23:26:41 christos Exp $ +# $File: compress,v 1.75 2019/04/19 00:42:27 christos Exp $ # compress: file(1) magic for pure-compression formats (no archives) # # compress, gzip, pack, compact, huf, squeeze, crunch, freeze, yabba, etc. @@ -16,13 +16,78 @@ >2 byte&0x1f x %d bits # gzip (GNU zip, not to be confused with Info-ZIP or PKWARE zip archiver) +# URL: https://en.wikipedia.org/wiki/Gzip +# Reference: https://tools.ietf.org/html/rfc1952 +# Update: Joerg Jenderek, Apr 2019 # Edited by Chris Chittleborough <cchittleborough@yahoo.com.au>, March 2002 # * Original filename is only at offset 10 if "extra field" absent # * Produce shorter output - notably, only report compression methods # other than 8 ("deflate", the only method defined in RFC 1952). -0 string \037\213 gzip compressed data -!:mime application/x-gzip -!:strength * 2 +# Note: find defs -iname '*.trid.xml' -exec grep -q '<Bytes>1F8B08' {} \; -ls +# TODO: +# FBR Blueberry FlashBack screen Record https://www.flashbackrecorder.com/ +# KPR KOffice/Calligra KPresenter application/x-kpresenter +# KPT KOffice/Calligra KPresenter template? application/x-kpresenter +# SAV Diggles Saved Game File http://www.innonics.com +# SAV FarCry (demo) saved game http://www.farcry-thegame.com +# DAT ZOAGZIP game data format http://en.wikipedia.org/wiki/SD_Gundam_Capsule_Fighter +0 string \037\213 +# to display gzip compressed (strength=100=2*50) before other (strength=50)? +#!:strength * 2 +# no FNAME and FCOMMENT bit implies no file name/comment. That means only binary +>3 byte&0x18 =0 +# For binary gzipped no ASCII text should occur +# mcd-monu-cad.trid.xml +>>10 string MCD Monu-Cad Drawing, Component or Font +#>>36 string Created\ with\ MONU-CAD +#!:mime application/octet-stream +# http://fileformats.archiveteam.org/wiki/Monu-CAD +# http://www.monucad.com/downloads/FullDemo-2005.EXE +# /HANDS96.MCC Component +# /DEMO_DD01.MCD Drawing +# /MCALF020.FNT Font +!:ext mcc/mcd/fnt +# http://www.generalcadd.com +>>10 string GXD General CADD, Drawing or Component +#!:mime application/octet-stream +# /gxc/BUILDINGEDGE.gxc Component +# /gxd/HOCKETT-STPAUL-WRHSE.gxd Drawing +# /gxd/POWERLAND-MILL-ADD-11.gxd Drawing v9.1.06 +!:ext gxc/gxd +#>>>13 ubyte 0 \b, version 0 +>>>13 string 09 \b, version 9 +# other gzipped binary like gzipped tar, VirtualBox extension package,... +>>10 default x gzip compressed data +>>>0 use gzip-info +# size of the original (uncompressed) input data modulo 2^32 +>>>-4 ulelong x \b, original size modulo 2^32 %u +# gzipped TAR or VirtualBox extension package +!:mime application/gzip +#!:mime application/x-compressed-tar +#!:mime application/x-virtualbox-vbox-extpack +# https://www.w3.org/TR/SVG/mimereg.html +#!:mime image/image/svg+xml-compressed +# zlib.3.gz +# microcode-20180312.tgz +# tpz same as tgz +# lua-md5_1.2-1_i386_i486.ipk https://en.wikipedia.org/wiki/Opkg +# Oracle_VM_VirtualBox_Extension_Pack-5.0.12-104815.vbox-extpack +!:ext gz/tgz/tpz/ipk/vbox-extpack/svgz +# FNAME/FCOMMENT bit implies file name/comment as iso-8859-1 text +>3 byte&0x18 >0 gzip compressed data +!:mime application/gzip +# gzipped tar, gzipped Abiword document +#!:mime application/x-compressed-tar +#!:mime application/x-abiword-compressed +#!:mime image/image/svg+xml-compressed +# kleopatra_splashscreen.svgz gzipped .svg +!:ext gz/tgz/tpz/zabw/svgz +>>0 use gzip-info +# size of the original (uncompressed) input data modulo 2^32 +>>-4 ulelong x \b, original size modulo 2^32 %u +# display information of gzip compressed files +0 name gzip-info +#>2 byte x THIS iS GZIP >2 byte <8 \b, reserved method >2 byte >8 \b, unknown method >3 byte &0x01 \b, ASCII @@ -49,7 +114,9 @@ >9 byte =0x0B \b, from NTFS filesystem (NT) >9 byte =0x0C \b, from QDOS >9 byte =0x0D \b, from Acorn RISCOS ->-4 lelong x \b, original size %d +# size of the original (uncompressed) input data modulo 2^32 +#>-4 ulelong x \b, original size modulo 2^32 %u +#ERROR: line 114: non zero offset 1048572 at level 1 # packed data, Huffman (minimum redundancy) codes on a byte-by-byte basis 0 string \037\036 packed data @@ -160,7 +227,7 @@ >>17 byte =0x0E os: Win32 # 4.3BSD-Quasijarus Strong Compression -# http://minnie.tuhs.org/Quasijarus/compress.html +# https://minnie.tuhs.org/Quasijarus/compress.html 0 string \037\241 Quasijarus strong compressed data # From: Cory Dikkers <cdikkers@swbell.net> @@ -174,7 +241,7 @@ >4 belong 0x090A0C0D best compression # 7-zip archiver, from Thomas Klausner (wiz@danbala.tuwien.ac.at) -# http://www.7-zip.org or DOC/7zFormat.txt +# https://www.7-zip.org or DOC/7zFormat.txt # 0 string 7z\274\257\047\034 7-zip archive data, >6 byte x version %d @@ -203,7 +270,7 @@ >5 byte x \b.%d !:mime application/x-lrzip -# http://fastcompression.blogspot.fi/2013/04/lz4-streaming-format-final.html +# https://fastcompression.blogspot.fi/2013/04/lz4-streaming-format-final.html 0 lelong 0x184d2204 LZ4 compressed data (v1.4+) !:mime application/x-lz4 # Added by osm0sis@xda-developers.com @@ -292,11 +359,11 @@ >0x8 lelong x \b, %u entries # Snappy framing format -# http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt +# https://code.google.com/p/snappy/source/browse/trunk/framing_format.txt 0 string \377\006\0\0sNaPpY snappy framed data !:mime application/x-snappy-framed -# qpress, http://www.quicklz.com/ +# qpress, https://www.quicklz.com/ 0 string qpress10 qpress compressed data !:mime application/x-qpress diff --git a/magic/Magdir/console b/magic/Magdir/console index ab2954f0b5af..5e5e5816ecc9 100644 --- a/magic/Magdir/console +++ b/magic/Magdir/console @@ -1,14 +1,14 @@ #------------------------------------------------------------------------------ -# $File: console,v 1.40 2018/06/23 16:40:40 christos Exp $ +# $File: console,v 1.45 2019/04/19 00:42:27 christos Exp $ # Console game magic # Toby Deshane <hac@shoelace.digivill.net> # ines: file(1) magic for Marat's iNES Nintendo Entertainment System ROM dump format # Updated by David Korth <gerbilsoft@gerbilsoft.com> # References: -# - http://wiki.nesdev.com/w/index.php/INES -# - http://wiki.nesdev.com/w/index.php/NES_2.0 +# - https://wiki.nesdev.com/w/index.php/INES +# - https://wiki.nesdev.com/w/index.php/NES_2.0 # Common header for iNES, NES 2.0, and Wii U iNES. 0 name nes-rom-image-ines @@ -57,7 +57,7 @@ #------------------------------------------------------------------------------ # unif: file(1) magic for UNIF-format Nintendo Entertainment System ROM images -# Reference: http://wiki.nesdev.com/w/index.php/UNIF +# Reference: https://wiki.nesdev.com/w/index.php/UNIF # From: David Korth <gerbilsoft@gerbilsoft.com> # # NOTE: The UNIF format uses chunks instead of a fixed header, @@ -69,7 +69,7 @@ #------------------------------------------------------------------------------ # fds: file(1) magic for Famciom Disk System disk images -# Reference: http://wiki.nesdev.com/w/index.php/Family_Computer_Disk_System#.FDS_format +# Reference: https://wiki.nesdev.com/w/index.php/Family_Computer_Disk_System#.FDS_format # From: David Korth <gerbilsoft@gerbilsoft.com> # TODO: Check "Disk info block" and get info from that in addition to the optional header. @@ -186,7 +186,7 @@ # genesis: file(1) magic for various Sega Mega Drive / Genesis ROM image and disc formats # Updated by David Korth <gerbilsoft@gerbilsoft.com> # References: -# - http://www.retrodev.com/segacd.html +# - https://www.retrodev.com/segacd.html # - http://devster.monkeeh.com/sega/32xguide1.txt # @@ -269,7 +269,7 @@ # Detects all Game Gear and export Sega Master System ROM images, # and some Japanese Sega Master System ROM images. # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://www.smspower.org/Development/ROMHeader +# Reference: https://www.smspower.org/Development/ROMHeader # # General SMS header rule. @@ -359,7 +359,7 @@ #------------------------------------------------------------------------------ # dreamcast: file(1) magic for the Sega Dreamcast disc image format. # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://mc.pp.se/dc/ip0000.bin.html +# Reference: https://mc.pp.se/dc/ip0000.bin.html # # Common Sega Dreamcast disc header format. @@ -422,7 +422,7 @@ #------------------------------------------------------------------------------ # gba: file(1) magic for the Nintendo Game Boy Advance raw ROM format -# Reference: http://problemkaputt.de/gbatek.htm#gbacartridgeheader +# Reference: https://problemkaputt.de/gbatek.htm#gbacartridgeheader # # Original version from: "Nelson A. de Oliveira" <naoliv@gmail.com> # Updated version from: David Korth <gerbilsoft@gerbilsoft.com> @@ -435,7 +435,7 @@ #------------------------------------------------------------------------------ # nds: file(1) magic for the Nintendo DS(i) raw ROM format -# Reference: http://problemkaputt.de/gbatek.htm#dscartridgeheader +# Reference: https://problemkaputt.de/gbatek.htm#dscartridgeheader # # Original version from: "Nelson A. de Oliveira" <naoliv@gmail.com> # Updated version from: David Korth <gerbilsoft@gerbilsoft.com> @@ -469,7 +469,7 @@ # From: David Korth <gerbilsoft@gerbilsoft.com> # References: # - https://neogpc.googlecode.com/svn-history/r10/trunk/src/core/neogpc.cpp -# - http://www.devrs.com/ngp/files/ngpctech.txt +# - https://www.devrs.com/ngp/files/ngpctech.txt # 0x0A string BY\ SNK\ CORPORATION Neo Geo Pocket !:mime application/x-neo-geo-pocket-rom @@ -507,7 +507,21 @@ #------------------------------------------------------------------------------ # Microsoft Xbox executables .xbe (Esa Hyytia <ehyytia@cc.hut.fi>) -0 string XBEH XBE, Microsoft Xbox executable +0 string XBEH Microsoft Xbox executable +# expect base address of 0x10000 +>0x0104 ulelong =0x10000 +>>(0x0118.l-0x0FFF4) lestring16 x \b: "%.40s" +>>(0x0118.l-0x0FFF5) byte x (%c +>>(0x0118.l-0x0FFF6) byte x \b%c- +>>(0x0118.l-0x0FFF8) uleshort x \b%03u) +>>(0x0118.l-0x0FF60) ulelong&0x80000007 0x80000007 \b, all regions +>>(0x0118.l-0x0FF60) ulelong&0x80000007 !0x80000007 +>>>(0x0118.l-0x0FF60) ulelong >0 (regions: +>>>>(0x0118.l-0x0FF60) ulelong &0x00000001 NA +>>>>(0x0118.l-0x0FF60) ulelong &0x00000002 Japan +>>>>(0x0118.l-0x0FF60) ulelong &0x00000004 Rest_of_World +>>>>(0x0118.l-0x0FF60) ulelong &0x80000000 Manufacturer +>>>(0x0118.l-0x0FF60) ulelong >0 \b) # probabilistic checks whether signed or not >0x0004 ulelong =0x0 >>&2 ulelong =0x0 @@ -515,22 +529,43 @@ >0x0004 ulelong >0 >>&2 ulelong >0 >>>&2 ulelong >0 \b, signed -# expect base address of 0x10000 ->0x0104 ulelong =0x10000 ->>(0x0118-0x0FF60) ulelong&0x80000007 0x80000007 \b, all regions ->>(0x0118-0x0FF60) ulelong&0x80000007 !0x80000007 ->>>(0x0118-0x0FF60) ulelong >0 (regions: ->>>>(0x0118-0x0FF60) ulelong &0x00000001 NA ->>>>(0x0118-0x0FF60) ulelong &0x00000002 Japan ->>>>(0x0118-0x0FF60) ulelong &0x00000004 Rest_of_World ->>>>(0x0118-0x0FF60) ulelong &0x80000000 Manufacturer ->>>(0x0118-0x0FF60) ulelong >0 \b) # -------------------------------- # Microsoft Xbox data file formats 0 string XIP0 XIP, Microsoft Xbox data 0 string XTF0 XTF, Microsoft Xbox data +#------------------------------------------------------------------------------ +# Microsoft Xbox 360 executables (.xex) +# From: David Korth <gerbilsoft@gerbilsoft.com> +# References: +# - https://free60project.github.io/wiki/XEX.html +# - https://github.com/xenia-project/xenia/blob/HEAD/src/xenia/kernel/util/xex2_info.h + +# Title ID (part of Execution ID section) +0 name xbox-360-xex-execution-id +>(0.L+0xC) byte x (%c +>(0.L+0xD) byte x \b%c +>(0.L+0xE) beshort x \b-%04u) + +0 string XEX2 Microsoft Xbox 360 executable +>0x18 search/0x100 \x00\x04\x00\x06 +>>&0 use xbox-360-xex-execution-id +>(0x010.L+0x178) ubelong 0xFFFFFFFF \b, all regions +>(0x010.L+0x178) ubelong !0xFFFFFFFF +>>(0x010.L+0x178) ubelong >0 (regions: +>>(0x010.L+0x178) ubelong&0x000000FF 0x000000FF USA +>>(0x010.L+0x178) ubelong&0x00000100 0x00000100 Japan +>>(0x010.L+0x178) ubelong&0x00000200 0x00000200 China +>>(0x010.L+0x178) ubelong&0x0000FC00 0x0000FC00 Asia +>>(0x010.L+0x178) ubelong&0x00FF0000 0x00FF0000 PAL +>>(0x010.L+0x178) ubelong&0x00FF0000 0x00FE0000 PAL [except AU/NZ] +>>(0x010.L+0x178) ubelong&0x00FF0000 0x00010000 AU/NZ +>>(0x010.L+0x178) ubelong&0xFF000000 0xFF000000 Other +>>(0x010.L+0x178) ubelong >0 \b) + + + # Atari Lynx cartridge dump (EXE/BLL header) # From: "Stefan A. Haubenthal" <polluks@web.de> @@ -563,7 +598,7 @@ >4 byte x %u tracks # IPS Patch Files from: From: Thomas Klausner <tk@giga.or.at> -# see http://zerosoft.zophar.net/ips.php +# see https://zerosoft.zophar.net/ips.php 0 string PATCH IPS patch file # Playstations Patch Files from: From: Thomas Klausner <tk@giga.or.at> @@ -654,7 +689,7 @@ # Type: Nintendo GameCube/Wii common disc header data. # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://wiibrew.org/wiki/Wii_Disc +# Reference: https://wiibrew.org/wiki/Wii_Disc 0 name nintendo-gcn-disc-common >0x20 string x "%.64s" >0x00 string x (%.6s @@ -668,7 +703,7 @@ # Type: Nintendo GameCube disc image # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://wiibrew.org/wiki/Wii_Disc +# Reference: https://wiibrew.org/wiki/Wii_Disc 0x1C belong 0xC2339F3D Nintendo GameCube disc image: !:mime application/x-gamecube-rom >0 use nintendo-gcn-disc-common @@ -685,13 +720,13 @@ # Type: Nintendo Wii disc image # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://wiibrew.org/wiki/Wii_Disc +# Reference: https://wiibrew.org/wiki/Wii_Disc 0x18 belong 0x5D1C9EA3 Nintendo Wii disc image: >0 use nintendo-gcn-disc-common # Type: Nintendo Wii disc image (WBFS format) # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://wiibrew.org/wiki/Wii_Disc +# Reference: https://wiibrew.org/wiki/Wii_Disc 0 string WBFS >0x218 belong 0x5D1C9EA3 Nintendo Wii disc image (WBFS format): !:mime application/x-wii-rom @@ -754,13 +789,15 @@ # Type: Nintendo GameCube/Wii disc image (with SDK header) # From: David Korth <gerbilsoft@gerbilsoft.com> -# Reference: http://wiibrew.org/wiki/Wii_Disc +# Reference: https://wiibrew.org/wiki/Wii_Disc 0 belong 0xFFFF0000 >0x18 belong 0x00000000 >>0x1C belong 0x00000000 >>>0x8018 belong 0x5D1C9EA3 Nintendo Wii SDK disc image: +!:mime application/x-wii-rom >>>>0x8000 use nintendo-gcn-disc-common >>>0x801C belong 0xC2339F3D Nintendo GameCube SDK disc image: +!:mime application/x-gamecube-rom >>>>0x8000 use nintendo-gcn-disc-common #------------------------------------------------------------------------------ diff --git a/magic/Magdir/coverage b/magic/Magdir/coverage index 6dd36fa61101..69eab704933c 100644 --- a/magic/Magdir/coverage +++ b/magic/Magdir/coverage @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: coverage,v 1.1 2016/06/05 00:26:32 christos Exp $ +# $File: coverage,v 1.2 2019/04/19 00:42:27 christos Exp $ # xoverage: file(1) magic for test coverage data # File formats used to store test coverage data @@ -67,7 +67,7 @@ # `clang -fprofile-instr-generate -fcoverage-mapping ...` # default name: default.profraw # magic is: \xFF lprofr \x81 -# cf. http://llvm.org/docs/doxygen/html/InstrProfData_8inc_source.html +# cf. https://llvm.org/docs/doxygen/html/InstrProfData_8inc_source.html 0 lequad 0xff6c70726f667281 LLVM raw profile data, >&0 byte x version %d @@ -78,10 +78,10 @@ # LLVM indexed instruction profile (as generated by llvm-profdata) # magic is: reverse(\xFF lprofi \x81) -# cf. http://llvm.org/docs/CoverageMappingFormat.html -# http://llvm.org/docs/doxygen/html/namespacellvm_1_1IndexedInstrProf.html -# http://llvm.org/docs/CommandGuide/llvm-cov.html -# http://llvm.org/docs/CommandGuide/llvm-profdata.html +# cf. https://llvm.org/docs/CoverageMappingFormat.html +# https://llvm.org/docs/doxygen/html/namespacellvm_1_1IndexedInstrProf.html +# https://llvm.org/docs/CommandGuide/llvm-cov.html +# https://llvm.org/docs/CommandGuide/llvm-profdata.html 0 lequad 0x8169666f72706cff LLVM indexed profile data, >&0 byte x version %d diff --git a/magic/Magdir/ctf b/magic/Magdir/ctf index 37fdd1b60d23..ebea8f316961 100644 --- a/magic/Magdir/ctf +++ b/magic/Magdir/ctf @@ -2,7 +2,7 @@ #-------------------------------------------------------------- # ctf: file(1) magic for CTF (Common Trace Format) trace files # -# Specs. available here: <http://www.efficios.com/ctf> +# Specs. available here: <https://www.efficios.com/ctf> #-------------------------------------------------------------- # CTF trace data diff --git a/magic/Magdir/cups b/magic/Magdir/cups index a065de39637d..6dd14ac5a5a2 100644 --- a/magic/Magdir/cups +++ b/magic/Magdir/cups @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: cups,v 1.5 2017/03/17 21:35:28 christos Exp $ +# $File: cups,v 1.6 2019/04/19 00:42:27 christos Exp $ # Cups: file(1) magic for the cups raster file format # From: Laurent Martelli <martellilaurent@gmail.com> -# http://www.cups.org/documentation.php/spec-raster.html +# https://www.cups.org/documentation.php/spec-raster.html # 0 name cups-le diff --git a/magic/Magdir/database b/magic/Magdir/database index dd88f25e9d7c..071a1156485b 100644 --- a/magic/Magdir/database +++ b/magic/Magdir/database @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: database,v 1.54 2018/06/23 16:37:21 christos Exp $ +# $File: database,v 1.55 2019/04/19 00:42:27 christos Exp $ # database: file(1) magic for various databases # # extracted from header/code files by Graeme Wilford (eep2gw@ee.surrey.ac.uk) @@ -147,8 +147,8 @@ ## XBase database files # updated by Joerg Jenderek at Feb 2013 -# http://www.dbase.com/Knowledgebase/INT/db7_file_fmt.htm -# http://www.clicketyclick.dk/databases/xbase/format/dbf.html +# https://www.dbase.com/Knowledgebase/INT/db7_file_fmt.htm +# https://www.clicketyclick.dk/databases/xbase/format/dbf.html # http://home.f1.htw-berlin.de/scheibl/db/intern/dBase.htm # inspect VVYYMMDD , where 1<= MM <= 12 and 1<= DD <= 31 0 ubelong&0x0000FFFF <0x00000C20 @@ -187,13 +187,13 @@ >>>>>>>>>>>>4 lelong >0 \b, %d record # plural s appended >>>>>>>>>>>>>4 lelong >1 \bs -# http://www.clicketyclick.dk/databases/xbase/format/dbf_check.html#CHECK_DBF +# https://www.clicketyclick.dk/databases/xbase/format/dbf_check.html#CHECK_DBF # 1 <= record size <= 4000 (dBase 3,4) or 32 * KB (=0x8000) >>>>>>>>>>>>10 uleshort x * %d # file size = records * record size + header size >>>>>>>>>>>>1 ubyte x \b, update-date >>>>>>>>>>>>1 use xbase-date -# http://msdn.microsoft.com/de-de/library/cc483186(v=vs.71).aspx +# https://msdn.microsoft.com/de-de/library/cc483186(v=vs.71).aspx #>>>>>>>>>>>>29 ubyte =0 \b, codepage ID=0x%x # 2~cp850 , 3~cp1252 , 0x1b~?? ; what code page is 0x1b ? >>>>>>>>>>>>29 ubyte >0 \b, codepage ID=0x%x @@ -261,13 +261,13 @@ # dBASE IV SQL, no memo;dbv memo var size (Flagship) >>0 ubyte 0x43 dBase IV, with SQL table !:mime application/x-dbf -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x62 dBase IV, with SQL table #!:mime application/x-dbf # dBASE IV, with memo!! >>0 ubyte 0x7b dBase IV, with memo !:mime application/x-dbf -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x82 dBase IV, with SQL system #!:mime application/x-dbf # FoxBase+/dBaseIII+ with memo .DBT! @@ -276,7 +276,7 @@ # VISUAL OBJECTS (first 1.0 versions) for the Dbase III files (NTX clipper driver); memo file >>0 ubyte 0x87 VISUAL OBJECTS, with memo file !:mime application/x-dbf -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x8A FoxBase+/dBase III, with memo .DBT #!:mime application/x-dbf # dBASE IV with memo! @@ -287,7 +287,7 @@ !:mime application/x-dbf # .dbv and .dbt memo (Flagship)? >>0 ubyte 0xB3 Flagship -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xCA dBase IV with memo .DBT #!:mime application/x-dbf # dBASE IV with SQL table, with memo .DBT @@ -296,12 +296,12 @@ # HiPer-Six format;Clipper SIX, with SMT memo file >>0 ubyte 0xE5 Clipper SIX with memo !:mime application/x-dbf -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xF4 dBase IV, with SQL table, with memo #!:mime application/x-dbf >>0 ubyte 0xF5 FoxPro with memo !:mime application/x-dbf -# http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx +# https://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xFA FoxPro 2.x, with memo #!:mime application/x-dbf # unknown version (should not happen) @@ -332,7 +332,7 @@ >>>>>2 ubyte x \b-%d # dBase memo files .DBT or .FPT -# http://msdn.microsoft.com/en-us/library/8599s21w(v=vs.80).aspx +# https://msdn.microsoft.com/en-us/library/8599s21w(v=vs.80).aspx 16 ubyte <4 >16 ubyte !2 >>16 ubyte !1 @@ -495,7 +495,7 @@ >>220 ulelong x \b.%d # From: Joerg Jenderek -# URL: http://forensicswiki.org/wiki/Windows_Application_Compatibility +# URL: https://forensicswiki.org/wiki/Windows_Application_Compatibility # Note: files contain application compatibility fixes, application compatibility modes and application help messages. 8 string sdbf >7 ubyte 0 @@ -542,7 +542,7 @@ >>7 byte x \b-%d # Type: Advanced Data Format (ADF) database -# URL: http://www.grc.nasa.gov/WWW/cgns/adf/ +# URL: https://www.grc.nasa.gov/WWW/cgns/adf/ # From: Nicolas Chauvat <nicolas.chauvat@logilab.fr> 0 string @(#)ADF\ Database CGNS Advanced Data Format @@ -582,7 +582,7 @@ 0 string ToKyO\ CaBiNeT\n TokyoCabinet database >14 string x (version %s) -# From: Stephane Blondon http://www.yaal.fr +# From: Stephane Blondon https://www.yaal.fr # Database file for Zope (done by FileStorage) 0 string FS21 Zope Object Database File Storage v3 (data) 0 string FS30 Zope Object Database File Storage v4 (data) @@ -593,7 +593,7 @@ # IDA (Interactive Disassembler) database 0 string IDA1 IDA (Interactive Disassembler) database -# Hopper (reverse engineering tool) http://www.hopperapp.com/ +# Hopper (reverse engineering tool) https://www.hopperapp.com/ 0 string hopperdb Hopper database # URL: https://en.wikipedia.org/wiki/Panorama_(database_engine) diff --git a/magic/Magdir/dataone b/magic/Magdir/dataone index 8bbe87aac141..8ef3f798163f 100644 --- a/magic/Magdir/dataone +++ b/magic/Magdir/dataone @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: dataone,v 1.1 2018/06/06 01:16:40 christos Exp $ +# $File: dataone,v 1.2 2019/04/19 00:42:27 christos Exp $ # # DataONE- files from Dave Vieglais <dave.vieglais@gmail.com> & # Pratik Shrivastava <pratikshrivastava23@gmail.com> @@ -13,35 +13,35 @@ >&0 regex (eml)-[0-9].[0-9].[0-9]+ eml://ecoinformatics.org/%s # onedcx (DataONE Dublin Core Extended v1.0) ->&0 regex (onedcx/v)[0-9].[0-9]+ http://ns.dataone.org/metadata/schema/onedcx/v1.0 +>&0 regex (onedcx/v)[0-9].[0-9]+ https://ns.dataone.org/metadata/schema/onedcx/v1.0 # FGDC-STD-001-1998 (Content Standard for Digital Geospatial Metadata, # version 001-1998) >&0 regex fgdc FGDC-STD-001-1998 # Mercury (Oak Ridge National Lab Mercury Metadata version 1.0) ->&0 regex (mercury/terms/v)[0-9].[0-9] http://purl.org/ornl/schema/mercury/terms/v1.0 +>&0 regex (mercury/terms/v)[0-9].[0-9] https://purl.org/ornl/schema/mercury/terms/v1.0 # ISOTC211 (Geographic MetaData (GMD) Extensible Markup Language) >&0 regex isotc211 ->>&0 regex eng;USA http://www.isotc211.org/2005/gmd +>>&0 regex eng;USA https://www.isotc211.org/2005/gmd # ISOTC211 (NOAA Variant Geographic MetaData (GMD) Extensible Markup Language) ->>&0 regex gov.noaa.nodc:[0-9]+ http://www.isotc211.org/2005/gmd-noaa +>>&0 regex gov.noaa.nodc:[0-9]+ https://www.isotc211.org/2005/gmd-noaa # ISOTC211 PANGAEA Variant Geographic MetaData (GMD) Extensible Markup Language ->>&0 regex pangaea.dataset[0-9][0-9][0-9][0-9][0-9][0-9]+ http://www.isotc211.org/2005/gmd-pangaea +>>&0 regex pangaea.dataset[0-9][0-9][0-9][0-9][0-9][0-9]+ https://www.isotc211.org/2005/gmd-pangaea !:mime text/xml # Object Reuse and Exchange Vocabulary 0 string <?xml >&0 regex rdf ->>&0 regex openarchives http://www.openarchives.org/ore/terms +>>&0 regex openarchives https://www.openarchives.org/ore/terms !:mime application/rdf+xml # Dryad Metadata Application Profile Version 3.1 0 string <DryadData ->&0 regex (dryad-bibo/v)[0-9].[0-9] http://datadryad.org/profile/v3.1 +>&0 regex (dryad-bibo/v)[0-9].[0-9] https://datadryad.org/profile/v3.1 !:mime text/xml diff --git a/magic/Magdir/dbpf b/magic/Magdir/dbpf index 8545720dae50..df07ff809a43 100644 --- a/magic/Magdir/dbpf +++ b/magic/Magdir/dbpf @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: dbpf,v 1.2 2018/05/24 18:54:40 christos Exp $ +# $File: dbpf,v 1.3 2019/04/19 00:42:27 christos Exp $ # dppf: Maxis Database Packed Files, the stored data file format used by all # Maxis games after the Sims: http://wiki.niotso.org/DBPF -# http://www.wiki.sc4devotion.com/index.php?title=DBPF +# https://www.wiki.sc4devotion.com/index.php?title=DBPF # 13 Oct 2017, Kip Warner <kip at thevertigo dot com> 0 string DBPF Maxis Database Packed File >4 ulelong x \b, version: %u. diff --git a/magic/Magdir/dolby b/magic/Magdir/dolby index 8034eddc2a8f..d73e7d35f999 100644 --- a/magic/Magdir/dolby +++ b/magic/Magdir/dolby @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: dolby,v 1.8 2017/03/17 21:35:28 christos Exp $ +# $File: dolby,v 1.9 2019/04/19 00:42:27 christos Exp $ # ATSC A/53 aka AC-3 aka Dolby Digital <ashitaka@gmx.at> -# from http://www.atsc.org/standards/a_52a.pdf +# from https://www.atsc.org/standards/a_52a.pdf # corrections, additions, etc. are always welcome! # # syncword diff --git a/magic/Magdir/dyadic b/magic/Magdir/dyadic index 185970e4ee00..c57f81b7cbbb 100644 --- a/magic/Magdir/dyadic +++ b/magic/Magdir/dyadic @@ -1,11 +1,11 @@ #------------------------------------------------------------------------------ -# $File: dyadic,v 1.8 2017/03/17 21:35:28 christos Exp $ +# $File: dyadic,v 1.9 2019/04/19 00:42:27 christos Exp $ # Dyadic: file(1) magic for Dyalog APL. # # updated by Joerg Jenderek at Oct 2013 -# http://en.wikipedia.org/wiki/Dyalog_APL -# http://www.dyalog.com/ +# https://en.wikipedia.org/wiki/Dyalog_APL +# https://www.dyalog.com/ # .DXV Dyalog APL External Variable # .DIN Dyalog APL Input Table # .DOT Dyalog APL Output Table diff --git a/magic/Magdir/ebml b/magic/Magdir/ebml index d5d174329a52..d37b5c0b23f1 100644 --- a/magic/Magdir/ebml +++ b/magic/Magdir/ebml @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: ebml,v 1.1 2010/07/02 00:07:03 christos Exp $ +# $File: ebml,v 1.2 2019/04/19 00:42:27 christos Exp $ # ebml: file(1) magic for various Extensible Binary Meta Language -# http://www.matroska.org/technical/specs/index.html#track +# https://www.matroska.org/technical/specs/index.html#track 0 belong 0x1a45dfa3 EBML file >4 search/b/100 \102\202 >>&1 string x \b, creator %.8s diff --git a/magic/Magdir/edid b/magic/Magdir/edid new file mode 100644 index 000000000000..a17b6c4ea728 --- /dev/null +++ b/magic/Magdir/edid @@ -0,0 +1,11 @@ + +#------------------------------------------------------------------------------ +# $File: edid,v 1.1 2019/03/28 12:36:01 christos Exp $ +# edid: file(1) magic for EDID dump files + +0 quad 0x00ffffffffffff00 Extended display identification data dump +!:mime application/x-edid-dump +>18 byte 0x01 Version 1 +>>19 byte <0x04 \b.%d +>18 byte 0x02 Version 2 +>>19 byte 0x00 \b.0 diff --git a/magic/Magdir/elf b/magic/Magdir/elf index 133bd1f37c3a..85eda78a2931 100644 --- a/magic/Magdir/elf +++ b/magic/Magdir/elf @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: elf,v 1.74 2018/06/23 16:39:53 christos Exp $ +# $File: elf,v 1.77 2019/01/16 19:33:35 christos Exp $ # elf: file(1) magic for ELF executables # # We have to check the byte order flag to see what byte order all the @@ -38,6 +38,8 @@ >0 lelong&0x3 2 relaxed memory ordering, 0 name elf-pa-risc +>2 leshort 0x0208 1.0 +>2 leshort 0x0210 1.1 >2 leshort 0x0214 2.0 >0 leshort &0x0008 (LP64) @@ -263,6 +265,8 @@ >18 leshort 217 iCelero CoolEngine, >18 leshort 218 Nanoradio Optimized RISC, >18 leshort 243 UCB RISC-V, +>18 leshort 247 eBPF, +>18 leshort 251 NEC VE, >18 leshort 0x1057 AVR (unofficial), >18 leshort 0x1059 MSP430 (unofficial), >18 leshort 0x1223 Adapteva Epiphany (unofficial), diff --git a/magic/Magdir/erlang b/magic/Magdir/erlang index b604a06828f1..df7aa2aac8ad 100644 --- a/magic/Magdir/erlang +++ b/magic/Magdir/erlang @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: erlang,v 1.6 2010/09/20 19:19:17 rrt Exp $ +# $File: erlang,v 1.7 2019/04/19 00:42:27 christos Exp $ # erlang: file(1) magic for Erlang JAM and BEAM files -# URL: http://www.erlang.org/faq/x779.html#AEN812 +# URL: https://www.erlang.org/faq/x779.html#AEN812 # OTP R3-R4 0 string \0177BEAM! Old Erlang BEAM file diff --git a/magic/Magdir/espressif b/magic/Magdir/espressif new file mode 100644 index 000000000000..72a0ec9b4f34 --- /dev/null +++ b/magic/Magdir/espressif @@ -0,0 +1,57 @@ + +# $File: espressif,v 1.1 2018/11/20 18:57:17 christos Exp $ +# configuration dump of Tasmota firmware for ESP8266 based devices by Espressif +# URL: https://github.com/arendst/Sonoff-Tasmota/ +# Reference: https://codeload.github.com/arendst/Sonoff-Tasmota/zip/release-6.2/ +# Sonoff-Tasmota-release-6.2.zip/Sonoff-Tasmota-release-6.2/sonoff/settings.h +# From: Joerg Jenderek +# +# cfg_holder=4617=0x1209 +0 uleshort 4617 +# remainig settings normally 0x5A+offset XORed; free_1D5[20] empty since 5.12.0e +>0x1D5 ubequad 0x2f30313233343536 configuration of Tasmota firmware (ESP8266) +!:mime application/x-tasmota-dmp +!:ext dmp +# version like 6.2.1.0 ~ 0x06020100 XORed to 0x63666262 +>>11 ubyte^0x65 x \b, version %u +>>10 ubyte^0x64 x \b.%u +>>9 ubyte^0x63 x \b.%u +>>8 ubyte^0x62 x \b.%u +#>8 ubelong x (0x%x) +# hostname[33] XORed +>>0x165 ubyte^0x1BF x \b, hostname %c +>>0x166 ubyte^0x1C0 >037 \b%c +>>0x167 ubyte^0x1C1 >037 \b%c +>>0x168 ubyte^0x1C2 >037 \b%c +>>0x169 ubyte^0x1C3 >037 \b%c +>>0x16A ubyte^0x1C4 >037 \b%c +>>0x16B ubyte^0x1C5 >037 \b%c +>>0x16C ubyte^0x1C6 >037 \b%c +>>0x16D ubyte^0x1C7 >037 \b%c +>>0x16E ubyte^0x1C8 >037 \b%c +>>0x16F ubyte^0x1C9 >037 \b%c +>>0x170 ubyte^0x1CA >037 \b%c +>>0x171 ubyte^0x1CB >037 \b%c +>>0x172 ubyte^0x1CC >037 \b%c +>>0x173 ubyte^0x1CD >037 \b%c +>>0x174 ubyte^0x1CE >037 \b%c +>>0x175 ubyte^0x1CF >037 \b%c +>>0x176 ubyte^0x1D0 >037 \b%c +>>0x177 ubyte^0x1D1 >037 \b%c +>>0x178 ubyte^0x1D2 >037 \b%c +>>0x179 ubyte^0x1D3 >037 \b%c +>>0x17A ubyte^0x1D4 >037 \b%c +>>0x17B ubyte^0x1D5 >037 \b%c +>>0x17C ubyte^0x1D6 >037 \b%c +>>0x17D ubyte^0x1D7 >037 \b%c +>>0x17E ubyte^0x1D8 >037 \b%c +>>0x17F ubyte^0x1D9 >037 \b%c +>>0x180 ubyte^0x1DA >037 \b%c +>>0x181 ubyte^0x1DB >037 \b%c +>>0x182 ubyte^0x1DC >037 \b%c +>>0x183 ubyte^0x1DD >037 \b%c +>>0x184 ubyte^0x1DE >037 \b%c +>>0x185 ubyte^0x1DF >037 \b%c +#>>0x165 string x (%.33s) + + diff --git a/magic/Magdir/esri b/magic/Magdir/esri index e070e4f83f45..e49a7ce40706 100644 --- a/magic/Magdir/esri +++ b/magic/Magdir/esri @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: esri,v 1.4 2009/09/19 16:28:09 christos Exp $ +# $File: esri,v 1.5 2019/04/19 00:42:27 christos Exp $ # ESRI Shapefile format (.shp .shx .dbf=DBaseIII) # Based on info from -# <URL:http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf> +# <URL:https://www.esri.com/library/whitepapers/pdfs/shapefile.pdf> 0 belong 9994 ESRI Shapefile >4 belong =0 >8 belong =0 diff --git a/magic/Magdir/filesystems b/magic/Magdir/filesystems index eb41868ff453..1920e562a67c 100644 --- a/magic/Magdir/filesystems +++ b/magic/Magdir/filesystems @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: filesystems,v 1.124 2018/01/12 12:35:30 christos Exp $ +# $File: filesystems,v 1.128 2019/04/23 15:43:27 christos Exp $ # filesystems: file(1) magic for different filesystems # 0 name partid @@ -268,21 +268,21 @@ # to display information (50) before DOS BPB (strength=70) and after DOS floppy (120) like in old file version !:strength +65 >2 string OSBS OS/BS MBR -# added by Joerg Jenderek at Feb 2013 according to http://thestarman.pcministry.com/asm/mbr/ -# and http://en.wikipedia.org/wiki/Master_Boot_Record +# added by Joerg Jenderek at Feb 2013 according to https://thestarman.pcministry.com/asm/mbr/ +# and https://en.wikipedia.org/wiki/Master_Boot_Record # test for nearly all MS-DOS Master Boot Record initial program loader (IPL) is now done by # characteristic assembler instructions: xor ax,ax;mov ss,ax;mov sp,7c00 >0 search/2 \x33\xc0\x8e\xd0\xbc\x00\x7c MS-MBR -# Microsoft Windows 95A and early ( http://thestarman.pcministry.com/asm/mbr/STDMBR.htm ) +# Microsoft Windows 95A and early ( https://thestarman.pcministry.com/asm/mbr/STDMBR.htm ) # assembler instructions: mov si,sp;push ax;pop es;push ax;pop ds;sti;cld >>8 ubequad 0x8bf45007501ffbfc -# http://thestarman.pcministry.com/asm/mbr/200MBR.htm +# https://thestarman.pcministry.com/asm/mbr/200MBR.htm >>>0x16 ubyte 0xF3 \b,DOS 2 >>>>219 regex Author\ -\ Author: # found "David Litton" , "A Pehrsson " >>>>>&0 string x "%s" >>>0x16 ubyte 0xF2 -# NEC MS-DOS 3.30 Rev. 3 . See http://thestarman.pcministry.com/asm/mbr/DOS33MBR.htm +# NEC MS-DOS 3.30 Rev. 3 . See https://thestarman.pcministry.com/asm/mbr/DOS33MBR.htm # assembler instructions: mov di,077c;cmp word ptrl[di],a55a;jnz >>>>0x22 ubequad 0xbf7c07813d5aa575 \b,NEC 3.3 # version MS-DOS 3.30 til MS-Windows 95A (WinVer=4.00.1111) @@ -314,7 +314,7 @@ # "Sistema operativo mancante" nn=0xe2 for italian version >>>>>0x79 ubyte >0 at offset 0x%x >>>>>>(0x79.b) string >\0 "%s" -# Microsoft Windows 95B to XP (http://thestarman.pcministry.com/asm/mbr/95BMEMBR.htm) +# Microsoft Windows 95B to XP (https://thestarman.pcministry.com/asm/mbr/95BMEMBR.htm) # assembler instructions: push ax;pop es;push ax;pop ds;cld;mov si,7c1b >>8 ubequad 0x5007501ffcbe1b7c # assembler instructions: rep;movsb;retf;mov si,07be;mov cl,04 @@ -341,7 +341,7 @@ # "\216\257\245\340\240\346\250\256\255\255\240\357 \341\250\341\342\245\254\240 \255\245 \255\240\251\244\245\255\240" nn=0x156 for russian version >>>>0xA9 ubyte x at offset 0x1%x >>>>(0xA9.b+0x100) string >\0 "%s" -# http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm +# https://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm # assembler instructions: rep;movsb;retf;mov BP,07be;mov cl,04 >>>24 ubequad 0xf3a4cbbdbe07b104 XP # where xxyyzz are lower bits from offsets of error messages varying for different languages @@ -362,7 +362,7 @@ # Microsoft Windows Vista or 7 # assembler instructions: ..;mov ds,ax;mov si,7c00;mov di,..00 >>8 ubequad 0xc08ed8be007cbf00 -# Microsoft Windows Vista (http://thestarman.pcministry.com/asm/mbr/VistaMBR.htm) +# Microsoft Windows Vista (https://thestarman.pcministry.com/asm/mbr/VistaMBR.htm) # assembler instructions: jnz 0729;cmp ebx,"TCPA" >>>0xEC ubequad 0x753b6681fb544350 Vista # where xxyyzz are lower bits from offsets of error messages varying for different languages @@ -380,7 +380,7 @@ # "Betriebssystem nicht vorhanden" zz=0x1?? for german version >>>>0x1b7 ubyte >0 at offset 0x1%x >>>>(0x1b7.b+0x100) string >\0 "%s" -# Microsoft Windows 7 (http://thestarman.pcministry.com/asm/mbr/W7MBR.htm) +# Microsoft Windows 7 (https://thestarman.pcministry.com/asm/mbr/W7MBR.htm) # assembler instructions: cmp ebx,"TCPA";cmp >>>0xEC ubequad 0x6681fb5443504175 Windows 7 # where xxyyzz are lower bits from offsets of error messages varying for different languages @@ -398,10 +398,10 @@ # "Betriebssystem nicht vorhanden" zz=0x1?? for german version >>>>0x1b7 ubyte >0 at offset 0x1%x >>>>(0x1b7.b+0x100) string >\0 "%s" -# http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DiskSigs -# http://en.wikipedia.org/wiki/MBR_disk_signature#ID +# https://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DiskSigs +# https://en.wikipedia.org/wiki/MBR_disk_signature#ID >>0x1b8 ulelong >0 \b, disk signature 0x%-.4x -# driveID/timestamp for Win 95B,98,98SE and ME. See http://thestarman.pcministry.com/asm/mbr/mystery.htm +# driveID/timestamp for Win 95B,98,98SE and ME. See https://thestarman.pcministry.com/asm/mbr/mystery.htm >>0xDA uleshort 0 >>>0xDC ulelong >0 \b, created # physical drive number (0x80-0xFF) when the Windows wrote that byte to the drive @@ -465,13 +465,13 @@ >>>>>420 ubyte x \b) # # SYSLINUX MBR moved -# http://www.acronis.de/ +# https://www.acronis.de/ >362 string MBR\ Error\ \0\r >>376 string ress\ any\ key\ to\040 >>>392 string boot\ from\ floppy...\0 \b, Acronis MBR # added by Joerg Jenderek -# http://www.visopsys.org/ -# http://partitionlogic.org.uk/ +# https://www.visopsys.org/ +# https://partitionlogic.org.uk/ >309 string No\ bootable\ partition\ found\r >>339 string I/O\ Error\ reading\ boot\ sector\r \b, Visopsys MBR >349 string No\ bootable\ partition\ found\r @@ -490,7 +490,7 @@ >>172 string LILO \b, version 22.5.8 Debian # updated by Joerg Jenderek at Oct 2008 # variables according to grub-0.97/stage1/stage1.S or -# http://www.gnu.org/software/grub/manual/grub.html#Embedded-data +# https://www.gnu.org/software/grub/manual/grub.html#Embedded-data # usual values are marked with comments to get only informations of strange GRUB loaders >342 search/60 \0Geom\0 #>0 ulelong x %x=0x009048EB , 0x2a9048EB 0 @@ -549,7 +549,7 @@ # skip FSInfosector >>>>>>0 string !RRaA # skip 3rd sector of MS x86 bootloader with assember instructions cli;MOVZX EAX,BYTE PTR [BP+10];MOV ECX, -# http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm +# https://thestarman.pcministry.com/asm/mbr/MSWIN41.htm >>>>>>>0 ubequad !0xfa660fb64610668b # skip 13rd sector of MS x86 bootloader >>>>>>>>0 ubequad !0x660fb64610668b4e @@ -562,7 +562,7 @@ >>>>>>>>>>>446 use partition-table # TODO: test for extended bootrecord (ebr) moved and merged with mbr partition table entries # mbr partition table entries end -# http://www.acronis.de/ +# https://www.acronis.de/ #FAT label=ACRONIS\ SZ #OEM-ID=BOOTWIZ0 >442 string Non-system\ disk,\040 @@ -995,8 +995,8 @@ >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s # added by Joerg Jenderek -# http://www.visopsys.org/ -# http://partitionlogic.org.uk/ +# https://www.visopsys.org/ +# https://partitionlogic.org.uk/ # OEM-ID=Visopsys >478 ulelong 0 >>(1.b+326) string I/O\ Error\ reading\040 @@ -1023,8 +1023,8 @@ #it prints a very short message ("RE") to the screen and hangs the computer. # x86 bootloader end -# added by Joerg Jenderek at Feb 2013 according to http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO -# and http://en.wikipedia.org/wiki/File_Allocation_Table#FS_Information_Sector +# added by Joerg Jenderek at Feb 2013 according to https://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO +# and https://en.wikipedia.org/wiki/File_Allocation_Table#FS_Information_Sector >0 string RRaA >>0x1E4 string rrAa \b, FSInfosector #>>0x1FC uleshort =0 SHOULD BE ZERO @@ -1087,7 +1087,7 @@ >11 ubyte x \b+ >11 use DOS-filename -# http://en.wikipedia.org/wiki/Master_boot_record#PTE +# https://en.wikipedia.org/wiki/Master_boot_record#PTE # display standard partition table 0 name partition-table #>0 ubyte x PARTITION-TABLE @@ -1154,20 +1154,20 @@ 0x18b string OS/2 OS/2 Boot Manager # updated by Joerg Jenderek at Oct 2008 and Sep 2012 -# http://syslinux.zytor.com/iso.php +# https://syslinux.zytor.com/iso.php # tested with versions 1.47,1.48,1.49,1.50,1.62,1.76,2.00,2.10;3.00,3.11,3.31,;3.70,3.71,3.73,3.75,3.80,3.82,3.84,3.86,4.01,4.03 and 4.05 # assembler instructions: cli;jmp 0:7Cyy (yy=0x40,0x5e,0x6c,0x6e,0x77);nop;nop 0 ulequad&0x909000007cc0eafa 0x909000007c40eafa >631 search/689 ISOLINUX\ isolinux Loader >>&0 string x (version %-4.4s) -# http://syslinux.zytor.com/pxe.php +# https://syslinux.zytor.com/pxe.php # assembler instructions: jmp 7C05 0 ulelong 0x007c05ea pxelinux loader (version 2.13 or older) # assembler instructions: pushfd;pushad 0 ulelong 0x60669c66 pxelinux loader # assembler instructions: jmp 05 0 ulelong 0xc00005ea pxelinux loader (version 3.70 or newer) -# http://syslinux.zytor.com/wiki/index.php/SYSLINUX +# https://syslinux.zytor.com/wiki/index.php/SYSLINUX 0 string LDLINUX\ SYS\ SYSLINUX loader >12 string x (older version %-4.4s) 0 string \r\nSYSLINUX\ SYSLINUX loader @@ -1213,7 +1213,7 @@ >0004 uleshort x # ERRorTeXT >>181 search/166 Error\ \0\r\n NetBSD mbr -# NT Drive Serial Number http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DS +# NT Drive Serial Number https://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DS >>>0x1B8 ubelong >0 \b,Serial 0x%-.8x # BOOTSEL definitions contains assembler instructions: int 0x13;pop dx;push dx;push dx >>>0xbb search/71 \xcd\x13\x5a\x52\x52 \b,bootselector @@ -1249,7 +1249,7 @@ >>>398 string Disk\ error\r\n\0FDD\0HDD\0 >>>>419 string \ EBIOS\r\n\0 AdvanceMAME mbr -# Neil Turton mbr loader variant of http://www.chiark.greenend.org.uk/~neilt/mbr/ +# Neil Turton mbr loader variant of https://www.chiark.greenend.org.uk/~neilt/mbr/ # added by Joerg Jenderek at Mar 2011 for versions 1.0.0 - 1.1.11 # for 1st version assembler instructions: cld;xor ax,ax;mov DS,ax;MOV ES,AX;mov SI, # or cld;xor ax,ax;mov SS,ax;XOR SP,SP;mov DS, @@ -1262,7 +1262,7 @@ >>>(0x1BC.s+9) ubyte x \bVersion<=%u #>>>(0x1BC.s+8) ubyte x asm_flag_%x >>>(0x1BC.s+8) ubyte&1 1 \b,Y2K-Fix -# variant used by testdisk of http://www.cgsecurity.org/wiki/Menu_MBRCode +# variant used by testdisk of https://www.cgsecurity.org/wiki/Menu_MBRCode >>>(0x1BC.s+8) ubyte&2 2 \b,TestDisk #0x1~1,..,0x8~4,0x10~F,0x80~A enabled #>>>(0x1BC.s+10) ubyte x \b,flags 0x%x @@ -1295,7 +1295,7 @@ # added by Joerg Jenderek # In the second sector (+0x200) are variables according to grub-0.97/stage2/asm.S or # grub-1.94/kern/i386/pc/startup.S -# http://www.gnu.org/software/grub/manual/grub.html#Embedded-data +# https://www.gnu.org/software/grub/manual/grub.html#Embedded-data # usual values are marked with comments to get only informations of strange GRUB loaders 0x200 uleshort 0x70EA # found only version 3.{1,2} @@ -1350,7 +1350,7 @@ # DOS x86 sector updated and separated from "DOS/MBR boot sector" by Joerg Jenderek at May 2011 # JuMP short bootcodeoffset NOP assembler instructions will usually be EB xx 90 # over BIOS parameter block (BPB) -# http://thestarman.pcministry.com/asm/2bytejumps.htm#FWD +# https://thestarman.pcministry.com/asm/2bytejumps.htm#FWD # older drives may use Near JuMP instruction E9 xx xx # minimal short forward jump found 0x29 for bootloaders or 0x0 # maximal short forward jump is 0x7f @@ -1403,7 +1403,7 @@ >>>>>26 ubyte =1 \b, heads %u # valid only for sector sizes with more then 32 Bytes >>>>>11 uleshort >32 -# http://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#Extended_BIOS_Parameter_Block +# https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#Extended_BIOS_Parameter_Block # skip for values 2,2Ah,70h,73h,DFh # and continue for extended boot signature values 0,28h,29h,80h >>>>>>38 ubyte&0x56 =0 @@ -1465,7 +1465,7 @@ # FAT32 bit specific >>>>>82 string/c fat32 \b, FAT (32 bit) >>>>>>36 ulelong x \b, sectors/FAT %u -# http://technet.microsoft.com/en-us/library/cc977221.aspx +# https://technet.microsoft.com/en-us/library/cc977221.aspx >>>>>>40 uleshort >0 \b, extension flags 0x%x #>>>>>>40 uleshort =0 \b, extension flags %hu >>>>>>42 uleshort >0 \b, fsVersion %u @@ -1482,7 +1482,7 @@ #>>>>>>50 uleshort =6 \b, Backup boot sector %u (usual) >>>>>>50 default x >>>>>>>50 uleshort x \b, Backup boot sector %u -# corrected by Joerg Jenderek at Feb 2011 according to http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO +# corrected by Joerg Jenderek at Feb 2011 according to https://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO >>>>>>52 ulelong >0 \b, reserved1 0x%x >>>>>>56 ulelong >0 \b, reserved2 0x%x >>>>>>60 ulelong >0 \b, reserved3 0x%x @@ -1508,7 +1508,7 @@ >>>>>>>(11.s) ulelong&0x00ffffF0 0x00ffffF0 \b, followed by FAT # floppy image !:mime application/x-ima -# NTFS specific added by Joerg Jenderek at Mar 2011 according to http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm +# NTFS specific added by Joerg Jenderek at Mar 2011 according to https://thestarman.pcministry.com/asm/mbr/NTFSBR.htm # and http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/bios-parameter-block.html # 0 FATs >>>>>16 ubyte =0 @@ -1539,18 +1539,39 @@ >>>>>>>>>72 ulequad x \b, serial number 0%llx >>>>>>>>>80 ulelong >0 \b, checksum 0x%x #>>>>>>>>>80 ulelong =0 \b, checksum 0x%x=0 (usual) ->>>>>>>>>0x258 ulelong&0x00009090 =0x00009090 ->>>>>>>>>>&-92 indirect x \b; contains -# For 2nd NTFS sector added by Joerg Jenderek at Jan 2013 -# http://thestarman.pcministry.com/asm/mbr/NTFSbrHexEd.htm -# unused assembler instructions JMP y2;NOP;NOP -0x056 ulelong&0xFFFF0FFF 0x909002EB -# unicode loadername terminated by CTRL-D ->(0.s*2) ulelong&0xFFFFFF00 0x00040000 +# unicode loadername size jump +>>>>>>>>>(0x200.s*2) ubyte x +# in next sector loadername terminated by unicode CTRL-D and $ +>>>>>>>>>>&0x1FF ulequad&0x0000FFffFFffFF00 0x0000002400040000 \b; contains +# if 2nd NTFS sectors is found then assume whole filesystem +#!:mime application/x-raw-disk-image +!:ext img/bin/ntfs +>>>>>>>>>>>0x200 use ntfs-sector2 + +# For 2nd NTFS sector added by Joerg Jenderek at Jan 2013, Mar 2019 +# https://thestarman.pcministry.com/asm/mbr/NTFSbrHexEd.htm +# unused assembler instructions short JMP y2;NOP;NOP +0x056 ulelong&0xFFFF0FFF 0x909002EB NTFS +#!:mime application/octet-stream +!:ext bin +>0 use ntfs-sector2 +# https://memory.dataram.com/products-and-services/software/ramdisk +# assembler instructions JMP C000;NOP +0x056 ulelong 0x9000c0e9 NTFS +#!:mime application/octet-stream +!:ext bin +>0 use ntfs-sector2 +# check for characteristics of second NTFS sector and then display loader name +0 name ntfs-sector2 +# number of utf16 characters of loadername +>0 uleshort <8 +# unused assembler instructions JMP y2;NOP;NOP or JMP C000;NOP +>>0x056 ulelong&0xFF0000FD 0x900000E9 # loadernames are NTLDR,CMLDR,PELDR,$LDR$ or BOOTMGR ->>0x002 lestring16 x Microsoft Windows XP/VISTA bootloader %-5.5s ->>0x12 string $ ->>>0x0c lestring16 x \b%-2.2s +>>>0x002 lestring16 x bootstrap %-5.5s +# check for 7 character length of loader name like BOOTMGR +>>>0 uleshort 7 +>>>>0x0c lestring16 x \b%-2.2s ### DOS,NTFS boot sectors end # ntfsclone-image is a special save format for NTFS volumes, @@ -1689,7 +1710,7 @@ # ext2/ext3 filesystems - Andreas Dilger <adilger@dilger.ca> # ext4 filesystem - Eric Sandeen <sandeen@sandeen.net> # volume label and UUID Russell Coker -# http://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ +# https://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ 0x438 leshort 0xEF53 Linux >0x44c lelong x rev %d >0x43e leshort x \b.%d @@ -1801,7 +1822,7 @@ ############################################################################ # Minix-ST kernel floppy 0x800 belong 0x46fc2700 Atari-ST Minix kernel image -# http://en.wikipedia.org/wiki/BIOS_parameter_block +# https://en.wikipedia.org/wiki/BIOS_parameter_block # floppies with valid BPB and any instruction at beginning >19 string \240\005\371\005\0\011\0\2\0 \b, 720k floppy >19 string \320\002\370\005\0\011\0\1\0 \b, 360k floppy @@ -1871,7 +1892,7 @@ # # all FAT12 (strength=70) floppies with sectorsize 512 added by Joerg Jenderek at Jun 2013 -# http://en.wikipedia.org/wiki/File_Allocation_Table#Exceptions +# https://en.wikipedia.org/wiki/File_Allocation_Table#Exceptions # Too Weak. #512 ubelong&0xE0ffff00 0xE0ffff00 # without valid Media descriptor in place of BPB, cases with are done at other places @@ -1884,7 +1905,7 @@ #>>512 ubyte =0xfa 320k #>>512 ubyte =0xfb 640k #>>512 ubyte =0xfc 180k -# look like an an old DOS directory entry +# look like an old DOS directory entry #>>>0xA0E ubequad 0 #>>>>0xA00 ubequad !0 #!:mime application/x-ima @@ -1915,26 +1936,26 @@ # only x86 short jump instruction found #>>>0 ubyte =0xEB #>>>>1 ubyte x \b, code offset 0x%x+2 -# http://thestarman.pcministry.com/DOS/ibm100/Boot.htm +# https://thestarman.pcministry.com/DOS/ibm100/Boot.htm # assembler instructions: CLI;MOV AX,CS;MOV DS,AX;MOV DX,0 #>>>>(1.b+2) ubequad 0xfa8cc88ed8ba0000 \b, PC-DOS 1.0 bootloader # ibmbio.com+ibmdos.com #>>>>>0x176 use DOS-filename #>>>>>0x181 ubyte x \b+ #>>>>>0x182 use DOS-filename -# http://thestarman.pcministry.com/DOS/ibm110/Boot.htm +# https://thestarman.pcministry.com/DOS/ibm110/Boot.htm # assembler instructions: CLI;MOV AX,CS;MOV DS,AX;XOR DX,DX;MOV #>>>>(1.b+2) ubequad 0xfa8cc88ed833d28e \b, PC-DOS 1.1 bootloader # ibmbio.com+ibmdos.com #>>>>>0x18b use DOS-filename #>>>>>0x196 ubyte x \b+ #>>>>>0x197 use DOS-filename -# http://en.wikipedia.org/wiki/Zenith_Data_Systems +# https://en.wikipedia.org/wiki/Zenith_Data_Systems # assembler instructions: MOV BX,07c0;MOV SS,BX;MOV SP,01c6 #>>>>(1.b+2) ubequad 0xbbc0078ed3bcc601 \b, Zenith Data Systems MS-DOS 1.25 bootloader # IO.SYS+MSDOS.SYS #>>>>>0x20 use 2xDOS-filename -# http://en.wikipedia.org/wiki/Corona_Data_Systems +# https://en.wikipedia.org/wiki/Corona_Data_Systems # assembler instructions: MOV AX,CS;MOV DS,AX;CLI;MOV SS,AX; #>>>>(1.b+2) ubequad 0x8cc88ed8fa8ed0bc \b, MS-DOS 1.25 bootloader # IO.SYS+MSDOS.SYS @@ -1987,7 +2008,7 @@ >307200 use cdrom # .cso files -# Reference: http://pismotec.com/ciso/ciso.h +# Reference: https://pismotec.com/ciso/ciso.h # NOTE: There are two other formats with the same magic but # completely incompatible specifications: # - GameCube/Wii CISO: https://github.com/dolphin-emu/dolphin/blob/master/Source/Core/DiscIO/CISOBlob.h @@ -1996,10 +2017,13 @@ # Other fields are used to determine what type of CISO this is: # - 0x04 == 0x00200000: GameCube/Wii CISO (block_size) # - 0x10 == 0x00000800: PSP CISO (ISO-9660 sector size) +# - 0x10 == 0x00004000: For >2GB files using maxcso... +# https://github.com/unknownbrackets/maxcso/issues/26 # - None of the above: Compact ISO. >4 lelong !0 >>4 lelong !0x200000 ->>>0x10 lelong !0x800 Compressed ISO CD image +>>>16 lelong !0x800 +>>>>16 lelong !0x4000 Compressed ISO CD image # cramfs filesystem - russell@coker.com.au 0 lelong 0x28cd3d45 Linux Compressed ROM File System data, little endian @@ -2247,7 +2271,7 @@ >984 string x volume label is '%-12.12s' # From: Thomas Klausner <wiz@NetBSD.org> -# http://filext.com/file-extension/DAA +# https://filext.com/file-extension/DAA # describes the daa file format. The magic would be: 0 string DAA\x0\x0\x0\x0\x0 PowerISO Direct-Access-Archive diff --git a/magic/Magdir/finger b/magic/Magdir/finger index d8611f685528..ab43ac6f9d7b 100644 --- a/magic/Magdir/finger +++ b/magic/Magdir/finger @@ -1,11 +1,11 @@ #------------------------------------------------------------------------------ -# $File: finger,v 1.2 2015/10/07 02:37:57 christos Exp $ +# $File: finger,v 1.3 2019/04/19 00:42:27 christos Exp $ # fingerprint: file(1) magic for fingerprint data # XPM bitmaps) # -# http://cgit.freedesktop.org/libfprint/libfprint/tree/libfprint/data.c +# https://cgit.freedesktop.org/libfprint/libfprint/tree/libfprint/data.c 0 string FP1 libfprint fingerprint data V1 >3 beshort x \b, driver_id %x diff --git a/magic/Magdir/flash b/magic/Magdir/flash index b48abe968eb4..33b734499c4d 100644 --- a/magic/Magdir/flash +++ b/magic/Magdir/flash @@ -1,12 +1,12 @@ #------------------------------------------------------------------------------ -# $File: flash,v 1.14 2017/05/25 20:09:55 christos Exp $ +# $File: flash,v 1.15 2019/04/19 00:42:27 christos Exp $ # flash: file(1) magic for Macromedia Flash file format # # See # -# http://www.macromedia.com/software/flash/open/ -# http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/\ +# https://www.macromedia.com/software/flash/open/ +# https://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/\ # en/devnet/swf/pdf/swf-file-format-spec.pdf page 27 # diff --git a/magic/Magdir/fonts b/magic/Magdir/fonts index cf9d58882371..f47736f4e4f4 100644 --- a/magic/Magdir/fonts +++ b/magic/Magdir/fonts @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: fonts,v 1.38 2017/11/14 15:48:36 christos Exp $ +# $File: fonts,v 1.41 2019/05/05 16:44:04 christos Exp $ # fonts: file(1) magic for font data # 0 search/1 FONT ASCII vfont text @@ -17,7 +17,7 @@ # Summary: PostScript Type 1 Printer Font Metrics # URL: https://en.wikipedia.org/wiki/PostScript_fonts -# Reference: http://partners.adobe.com/public/developer/en/font/5178.PFM.pdf +# Reference: https://partners.adobe.com/public/developer/en/font/5178.PFM.pdf # Modified by: Joerg Jenderek # Note: moved from ./msdos magic # dfVersion 256=0100h @@ -80,7 +80,7 @@ 0 search/1 STARTFONT\ X11 BDF font text # From: Joerg Jenderek -# URL: http://grub.gibibit.com/New_font_format +# URL: https://grub.gibibit.com/New_font_format # Reference: util/grub-mkfont.c # include/grub/fontformat.h # FONT_FORMAT_SECTION_NAMES_FILE @@ -128,6 +128,58 @@ 7 belong 0x00564944 DOS code page font data (from Linux?) 4098 string DOSFONT DOSFONT2 encrypted font data +# From: Joerg Jenderek +# URL: http://fileformats.archiveteam.org/wiki/GEM_bitmap_font +# Reference: http://cd.textfiles.com/ataricompendium/BOOK/HTML/APPENDC.HTM#cnt +# +# usual case with lightening mask and skewing mask 5555h~UU +62 ulelong 0x55555555 +>0 use gdos-font +# BOX18.GFT COWBOY30.GFT ROYALK30.GFT +62 ulelong 0 +# skip ISO 9660 CD-ROM ./filesystem by looking for low positive face size +>2 uleshort >2 +# skip DOS 2.0 backup id file ./msdos by looking for face size lower/equal 48 +>>2 uleshort <49 +# skip MS Windows ICO ./msdos by looking for valid face name +>>>4 ubeshort >0x1F00 +# skip DOS executable BACKM212.COM by looking for horizontal offset table after header +#>>>>68 ulelong >87 OFFSET_OK +>>>>0 use gdos-font +0 name gdos-font +>0 uleshort x GEM GDOS font +!:mime application/x-font-gdos +# also .eps found like AA070GEP.EPS AI360GEP.EPS +!:ext fnt/gtf +# font name like University Bold +>4 string x %.32s +# face size in points 3-48 +>2 uleshort x %u +# face ID (must be unique) +>0 uleshort x \b, ID 0x%4.4x +# lowest character index in face (usually 32 for disk-loaded fonts). +#>36 uleshort x \b, low character index %u +# width of the widest character +#>50 uleshort x \b, %u char width +# width of the widest character cell +#>52 uleshort x \b, %u cell width +# thickening size +#>58 uleshort x \b, %u thick +# lightening mask to eliminate pixels, usually 5555h +>62 uleshort !0x5555 \b, lightening mask 0x%x +# skewing mask to determine when to perform additional rotation when skewing, usually 5555h +>64 uleshort !0x5555 \b, skewing mask 0x%x +# offset to horizontal offset table 58h~88 5eh +#>68 ulelong >88 \b, 0x%x horizontal table offset +# offset character offset table +#>72 ulelong x \b, 0x%x coffset +# offset to font data +#>72 ulelong x \b, 0x%x foffset +# form width in bytes +#>80 uleshort x \b, %u fwidth +# pointer to the next font, set by GDOS after loading +#>84 ulelong x \b, 0x%x noffset + # downloadable fonts for browser (prints type) anthon@mnt.org # https://tools.ietf.org/html/rfc3073 0 string PFR1 Portable Font Resource font data (new) @@ -167,16 +219,14 @@ >>12 regex/4l \^[A-Za-z][A-Za-z][A-Za-z/][A-Za-z2\ ] #>>>0 ubelong x \b, sfnt version 0x%x >>>0 ubelong !0x4f54544f TrueType -!:mime application/font-sfnt -#!:mime font/ttf +!:mime font/sfnt !:apple ????tfil # .ttf for TrueType font # EUDC.tte created by privat character editor %WINDIR%\system32\eudcedit.exe !:ext ttf/tte # sfnt version 4F54544Fh~OTTO >>>0 ubelong =0x4f54544f OpenType -!:mime application/font-sfnt -#!:mime font/otf +!:mime font/otf !:apple ????OTTO !:ext otf >>>0 ubelong x Font data @@ -257,7 +307,7 @@ # TrueType/OpenType font collections (.ttc) # URL: https://en.wikipedia.org/wiki/OpenType -# http://www.microsoft.com/typography/otspec/otff.htm +# https://www.microsoft.com/typography/otspec/otff.htm # Modified by: Joerg Jenderek # Note: container for TrueType, OpenType font 0 string ttcf @@ -265,17 +315,16 @@ >4 ubyte 0 # sfnt version often 0x00010000 of 1st table is TrueType >>(12.L) ubelong !0x4f54544f TrueType -#!:mime font/ttf +!:mime font/ttf !:apple ????tfil !:ext ttc # sfnt version 4F54544Fh~OTTO of 1st table is OpenType font >>(12.L) ubelong =0x4f54544f OpenType -#!:mime font/otf +!:mime font/otf !:apple ????OTTO # no example found for otc !:ext ttc/otc >>4 ubyte x font collection data -!:mime application/font-sfnt #!:mime font/collection # TCC version >>4 belong 0x00010000 \b, 1.0 @@ -320,12 +369,12 @@ #>14 beshort x \b, reserved %d #>16 belong x \b, totalSfntSize %d -# http://www.w3.org/TR/WOFF/ +# https://www.w3.org/TR/WOFF/ 0 string wOFF Web Open Font Format >0 use woff >20 beshort x \b, version %d >22 beshort x \b.%d -# http://www.w3.org/TR/WOFF2/ +# https://www.w3.org/TR/WOFF2/ 0 string wOF2 Web Open Font Format (Version 2) >0 use woff #>20 belong x \b, totalCompressedSize %d diff --git a/magic/Magdir/fsav b/magic/Magdir/fsav index d260157bc3f0..7ea094144e70 100644 --- a/magic/Magdir/fsav +++ b/magic/Magdir/fsav @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: fsav,v 1.15 2018/07/16 12:30:41 christos Exp $ +# $File: fsav,v 1.19 2019/04/19 00:42:27 christos Exp $ # fsav: file(1) magic for datafellows fsav virus definition files # Anthon van der Neut (anthon@mnt.org) @@ -40,25 +40,64 @@ #>>>>>>13 ubyte x \b%02x bytes # Joerg Jenderek: joerg dot jenderek at web dot de -# http://www.clamav.net/doc/latest/html/node45.html -# .cvd files start with a 512 bytes colon separated header +# clamav-0.100.2\docs\html\node60.html +# https://github.com/vrtadmin/clamav-faq/raw/master/manual/clamdoc.pdf +# ClamAV virus database files start with a 512 bytes colon separated header # ClamAV-VDB:buildDate:version:signaturesNumbers:functionalityLevelRequired:MD5:Signature:builder:buildTime -# + gzipped tarball files -0 string ClamAV-VDB: ->11 string >\0 Clam AntiVirus database %-.23s ->>34 string : ->>>35 string !: \b, version ->>>>35 string x \b %-.1s ->>>>>36 string !: ->>>>>>36 string x \b%-.1s ->>>>>>>37 string !: ->>>>>>>>37 string x \b%-.1s ->>>>>>>>>38 string !: ->>>>>>>>>>38 string x \b%-.1s ->>>>>>>>>>>39 string !: ->>>>>>>>>>>>39 string x \b%-.1s ->512 string \037\213 \b, gzipped ->769 string ustar\0 \b, tarred +# + gzipped (optional) tarball files +# output can often be verified by `sigtool --info=FILE` +0 string ClamAV-VDB: Clam AntiVirus +# padding spaces implies database +>511 ubyte =0x20 database +!:mime application/x-clamav-database +# empty build time +>>10 string =:: (unsigned) +# sigtool(1) man page +!:ext cud +# display some text to avoid error like: +# Magdir/fsav, 78: Warning: Current entry does not yet have a description for adding a EXTENSION type +# file: could not find any valid magic files! (No error) +>>10 default x (with buildtime) +#>>10 default x +# clamtmp is used for temporily database like update process +# for pure tar database only cld extension found +!:ext cld/cvd/clamtmp/cud +>511 default x file +!:mime application/x-clamav +!:ext info +>11 string >\0 +# buildDate empty or like "22 Mar 2017 12-57 -0400"; verified by `sigtool -i FILE` +>>11 regex \^[^:]{0,23} \b, %s +# version like 25170 +>>>&1 regex \^[^:]{1,6} \b, version %s +# signaturesNumbers like 4566249 +>>>>&1 regex \^[^:]{1,10} \b, %s signatures +# functionalityLevelRequired like 60 +>>>>>&1 regex \^[^:]{1,4} \b, level %s +# X for nothing or MD5 +#>>>>>>&1 regex \^[^:]{1,32} \b, MD5 "%s" +>>>>>>&1 regex \^[^:]{1,32} +# X for nothing or digital signature starting like AIzk/LYbX +#>>>>>>>&1 regex \^[^:]{1,255} \b, signature "%s" +>>>>>>>&1 regex \^[^:]{1,255} +# builder like neo +>>>>>>>>&1 regex \^[^:]{1,32} \b, builder %s +# buildTime like 1506611558 +#>>>>>>>>>&1 regex \^[^:]{1,10} \b, %s +>>>>>>>>>&1 regex \^[^:]{1,10} +# padding with spaces +#>>>>>>>>>>&1 ubequad x \b, padding 0x%16.16llx +>510 ubyte =0x20 +# inspect real database content +#>>512 ubeshort x \b, database MAGIC 0x%x +# ./archive handle pure tar archives +>>1012 quad =0 \b, with +>>>512 use tar-file +# not pure tar +>>1012 quad !0 +# one space at the end of text and then handles gziped archives by ./compress +>>>512 string \037\213 \b, with +>>>>512 indirect x # Type: Grisoft AVG AntiVirus # From: David Newgas <david@newgas.net> @@ -66,3 +105,24 @@ 0 string X5O!P%@AP[4\\PZX54(P^)7CC)7}$EICAR >33 string -STANDARD-ANTIVIRUS-TEST-FILE!$H+H* EICAR virus test files + +# From: Joerg Jenderek +# URL: https://www.avira.com/ +# Note: found in directory %ProgramData%\Avira\Antivirus\INFECTED (Windows) +# tested with version 15.0.43.23 at November 2019 +0 string AntiVir\ Qua Avira AntiVir quarantined +!:mime application/x-avira-qua +#!:mime application/octet-stream +!:ext qua +>156 string SUSPICIOUS_FILE +# file path of suspicious file +>>220 lestring16 x %s +>156 string !SUSPICIOUS_FILE +# file path of virus file +>>228 lestring16 x %s +# quarantined date +>60 ldate x at %s +# virus/danger name +>156 string !SUSPICIOUS_FILE +>>156 string x \b, category "%s" + diff --git a/magic/Magdir/games b/magic/Magdir/games index c5a5e4d041aa..30e6159a1746 100644 --- a/magic/Magdir/games +++ b/magic/Magdir/games @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: games,v 1.16 2017/10/19 16:40:37 christos Exp $ +# $File: games,v 1.17 2019/04/19 00:42:27 christos Exp $ # games: file(1) for games # Fabio Bonelli <fabiobonelli@libero.it> @@ -211,7 +211,7 @@ # Summary: SGF Smart Game Format # Extension: .sgf -# Reference: http://www.red-bean.com/sgf/ +# Reference: https://www.red-bean.com/sgf/ # Created by: Eduardo Sabbatella <eduardo_sabbatella@yahoo.com.ar> # Modified by (1): Abel Cheung (regex, more game format) # FIXME: Some games don't have GM (game type) @@ -281,7 +281,7 @@ >>&0 regex [0-9a-z.]+ \b, version %s # Type: SGF Smart Game Format -# URL: http://www.red-bean.com/sgf/ +# URL: https://www.red-bean.com/sgf/ # From: Eduardo Sabbatella <eduardo_sabbatella@yahoo.com.ar> 2 regex/c \\(;.*GM\\[[0-9]{1,2}\\] Smart Game Format >2 regex/c GM\\[1\\] - Go Game diff --git a/magic/Magdir/geo b/magic/Magdir/geo index 3e75b703deaf..d72e514a2338 100644 --- a/magic/Magdir/geo +++ b/magic/Magdir/geo @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: geo,v 1.6 2018/03/11 00:48:16 christos Exp $ +# $File: geo,v 1.7 2019/04/19 00:42:27 christos Exp $ # Geo- files from Kurt Schwehr <schwehr@ccom.unh.edu> ###################################################################### @@ -49,7 +49,7 @@ ###################################################################### # -# MULTIBEAM SONARS http://www.ldeo.columbia.edu/res/pi/MB-System/formatdoc/ +# MULTIBEAM SONARS https://www.ldeo.columbia.edu/res/pi/MB-System/formatdoc/ # ###################################################################### @@ -67,11 +67,11 @@ # format. It is the format of our upgraded SeaBeam 2112 on R/V KNORR. 0 string $HSF XSE multibeam -# mb121 http://www.saic.com/maritime/gsf/ +# mb121 https://www.saic.com/maritime/gsf/ 8 string GSF-v SAIC generic sensor format (GSF) sonar data, >&0 regex [0-9]*\.[0-9]* version %s -# MGD77 - http://www.ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.htm +# MGD77 - https://www.ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.htm # mb161 9 string MGD77 MGD77 Header, Marine Geophysical Data Exchange Format @@ -104,7 +104,7 @@ # https://midas.psi.ch/elog/ 0 string $@MID@$ elog journal entry -# Geospatial Designs http://www.geospatialdesigns.com/surfer6_format.htm +# Geospatial Designs https://www.geospatialdesigns.com/surfer6_format.htm 0 string DSBB Surfer 6 binary grid file >4 leshort x \b, %d >6 leshort x \bx%d @@ -117,7 +117,7 @@ # magic for LAS format files # alex myczko <alex@aiei.ch> -# http://www.asprs.org/wp-content/uploads/2010/12/LAS_1_3_r11.pdf +# https://www.asprs.org/wp-content/uploads/2010/12/LAS_1_3_r11.pdf 0 string LASF LIDAR point data records >24 byte >0 \b, version %u >25 byte >0 \b.%u diff --git a/magic/Magdir/glibc b/magic/Magdir/glibc new file mode 100644 index 000000000000..3b856f38363f --- /dev/null +++ b/magic/Magdir/glibc @@ -0,0 +1,21 @@ + +#------------------------------------------------------------------------------ +# $File: glibc,v 1.1 2018/10/11 15:35:43 christos Exp $ +# glibc locale files +# +# https://sourceware.org/git/?p=glibc.git;f=locale/localeinfo.h;h=68822a63#l32 + +0 belong 0x20070920 glibc locale file LC_CTYPE +0 belong 0x14110320 glibc locale file LC_NUMERIC +0 belong 0x17110320 glibc locale file LC_TIME +0 belong 0x17100520 glibc locale file LC_COLLATE +0 belong 0x11110320 glibc locale file LC_MONETARY +0 belong 0x10110320 glibc locale file LC_MESSAGES +0 belong 0x13110320 glibc locale file LC_ALL +0 belong 0x12110320 glibc locale file LC_PAPER +0 belong 0x1d110320 glibc locale file LC_NAME +0 belong 0x1c110320 glibc locale file LC_ADDRESS +0 belong 0x1f110320 glibc locale file LC_TELEPHONE +0 belong 0x1e110320 glibc locale file LC_MEASUREMENT +0 belong 0x19110320 glibc locale file LC_IDENTIFICATION + diff --git a/magic/Magdir/gnome b/magic/Magdir/gnome index 32c9ae938812..2905340c7e06 100644 --- a/magic/Magdir/gnome +++ b/magic/Magdir/gnome @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: gnome,v 1.5 2014/04/30 21:41:02 christos Exp $ +# $File: gnome,v 1.6 2019/04/19 00:42:27 christos Exp $ # GNOME related files # Contributed by Josh Triplett @@ -42,7 +42,7 @@ # It's always "GVariant", it's byte swapped on incompatible archs # See https://github.com/GNOME/gvdb/blob/master/gvdb-builder.c # file_builder_serialise() -# http://developer.gnome.org/glib/2.34/glib-GVariant.html#GVariant +# https://developer.gnome.org/glib/2.34/glib-GVariant.html#GVariant 0 string GVariant GVariant Database file, # version is never filled. probably future extension >8 lelong x version %d @@ -51,7 +51,7 @@ #>>>20 lelong x end %d # G-IR database made by gobject-introspect toolset, -# http://live.gnome.org/GObjectIntrospection +# https://live.gnome.org/GObjectIntrospection 0 string GOBJ\nMETADATA\r\n\032 G-IR binary database >16 byte x \b, v%d >17 byte x \b.%d diff --git a/magic/Magdir/gnu b/magic/Magdir/gnu index f30f8d7d9b8f..bf6b93da94d0 100644 --- a/magic/Magdir/gnu +++ b/magic/Magdir/gnu @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: gnu,v 1.20 2018/02/24 16:11:23 christos Exp $ +# $File: gnu,v 1.21 2019/04/19 00:42:27 christos Exp $ # gnu: file(1) magic for various GNU tools # # GNU nlsutils message catalog file format @@ -148,7 +148,7 @@ # GnuPG Keybox file -# <http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=kbx/keybox-blob.c;hb=HEAD> +# <https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=kbx/keybox-blob.c;hb=HEAD> # From: Philipp Hahn <hahn@univention.de> 0 belong 32 >4 byte 1 diff --git a/magic/Magdir/graphviz b/magic/Magdir/graphviz index cddc1164005a..d8bf22db4353 100644 --- a/magic/Magdir/graphviz +++ b/magic/Magdir/graphviz @@ -1,7 +1,7 @@ #------------------------------------------------------------------------------ -# $File: graphviz,v 1.8 2014/06/03 19:01:34 christos Exp $ -# graphviz: file(1) magic for http://www.graphviz.org/ +# $File: graphviz,v 1.9 2019/04/30 04:01:40 christos Exp $ +# graphviz: file(1) magic for https://www.graphviz.org/ # FIXME: These patterns match too generally. For example, the first # line matches a LaTeX file containing the word "graph" (with a { diff --git a/magic/Magdir/guile b/magic/Magdir/guile index 7ee0c48b2c8b..99f837133c7c 100644 --- a/magic/Magdir/guile +++ b/magic/Magdir/guile @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: guile,v 1.1 2011/12/16 17:44:33 christos Exp $ +# $File: guile,v 1.2 2019/04/19 00:42:27 christos Exp $ # Guile file magic from <dalepsmith@gmail.com> -# http://www.gnu.org/s/guile/ -# http://git.savannah.gnu.org/gitweb/?p=guile.git;f=libguile/_scm.h;hb=HEAD#l250 +# https://www.gnu.org/s/guile/ +# https://git.savannah.gnu.org/gitweb/?p=guile.git;f=libguile/_scm.h;hb=HEAD#l250 0 string GOOF---- Guile Object >8 string LE \b, little endian diff --git a/magic/Magdir/hardware b/magic/Magdir/hardware new file mode 100644 index 000000000000..e92986c5a81c --- /dev/null +++ b/magic/Magdir/hardware @@ -0,0 +1,12 @@ + +#------------------------------------------------------------------------------ +# $File: hardware,v 1.1 2018/08/02 06:32:52 christos Exp $ +# hardware magic + +# EDID +# https://en.wikipedia.org/wiki/Extended_Display_Identification_Data +0 string \x00\xFF\xFF\xFF\xFF\xFF\xFF\x00 +>19 byte x +>>18 byte x EDID data, version %u. +>>19 byte x \b%u +#>>17 ubyte+1990 <255 \b, manufactured %u diff --git a/magic/Magdir/hitachi-sh b/magic/Magdir/hitachi-sh index 0238ed7ebc4e..18d83844515f 100644 --- a/magic/Magdir/hitachi-sh +++ b/magic/Magdir/hitachi-sh @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: hitachi-sh,v 1.8 2017/03/17 21:35:28 christos Exp $ +# $File: hitachi-sh,v 1.9 2018/08/21 12:48:41 christos Exp $ # hitach-sh: file(1) magic for Hitachi Super-H # # Super-H COFF @@ -18,6 +18,7 @@ # use big endian variant of subroutine to display name+variables+flags # for common object formated files >>0 use \^display-coff +!:strength -10 0 leshort 0x0550 # test for unused flag bits in f_flags @@ -25,4 +26,5 @@ # use little endian variant of subroutine to # display name+variables+flags for common object formated files >>0 use display-coff +!:strength -10 diff --git a/magic/Magdir/hp b/magic/Magdir/hp index b01c3a40b0a7..d57169ee16e7 100644 --- a/magic/Magdir/hp +++ b/magic/Magdir/hp @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: hp,v 1.24 2014/04/30 21:41:02 christos Exp $ +# $File: hp,v 1.25 2019/01/13 00:32:38 christos Exp $ # hp: file(1) magic for Hewlett Packard machines (see also "printer") # # XXX - somebody should figure out whether any byte order needs to be @@ -120,12 +120,6 @@ 0 belong 0x020b010d PA-RISC1.0 dynamic load library >96 belong >0 - not stripped -0 belong 0x213c6172 archive file ->68 belong 0x020b0619 - PA-RISC1.0 relocatable library ->68 belong 0x02100619 - PA-RISC1.1 relocatable library ->68 belong 0x02110619 - PA-RISC1.2 relocatable library ->68 belong 0x02140619 - PA-RISC2.0 relocatable library - #### 500 0 long 0x02080106 HP s500 relocatable executable >16 long >0 - version %d diff --git a/magic/Magdir/ibm6000 b/magic/Magdir/ibm6000 index 8b48f370eb79..2112e71652d3 100644 --- a/magic/Magdir/ibm6000 +++ b/magic/Magdir/ibm6000 @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: ibm6000,v 1.13 2017/03/17 21:35:28 christos Exp $ +# $File: ibm6000,v 1.14 2019/03/07 17:21:54 christos Exp $ # ibm6000: file(1) magic for RS/6000 and the RT PC. # 0 beshort 0x01df executable (RISC System/6000 V3.1) or obj module @@ -17,6 +17,8 @@ 0 belong 0x000001f9 AIX compiled message catalog 0 string \<aiaff> archive 0 string \<bigaf> archive (big format) +0 belong 0x09006bea AIX backup/restore format file +0 belong 0x09006fea AIX backup/restore format file 0 beshort 0x01f7 64-bit XCOFF executable or object module >20 belong 0 not stripped diff --git a/magic/Magdir/images b/magic/Magdir/images index 15ec18a70d05..0e314adecb18 100644 --- a/magic/Magdir/images +++ b/magic/Magdir/images @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: images,v 1.146 2018/07/03 20:55:37 christos Exp $ +# $File: images,v 1.160 2019/04/19 00:42:27 christos Exp $ # images: file(1) magic for image formats (see also "iff", and "c-lang" for # XPM bitmaps) # @@ -27,23 +27,25 @@ # and Image Type 1 2 3 9 10 11 32 33 # and Color Map Entry Size 0 15 16 24 32 0 ubequad&0x00FeC400000000C0 0 +# Prevent conflicts with CRI ADX. +>(2.S-2) belong !0x28632943 # skip more garbage like *.iso by looking for positive image type ->2 ubyte >0 +>>2 ubyte >0 # skip some compiled terminfo like xterm+tmux by looking for image type less equal 33 ->>2 ubyte <34 +>>>2 ubyte <34 # skip arches.3200 , Finder.Root , Slp.1 by looking for low pixel depth 1 8 15 16 24 32 ->>>16 ubyte 1 ->>>>0 use tga-image ->>>16 ubyte 8 ->>>>0 use tga-image ->>>16 ubyte 15 ->>>>0 use tga-image ->>>16 ubyte 16 ->>>>0 use tga-image ->>>16 ubyte 24 ->>>>0 use tga-image ->>>16 ubyte 32 ->>>>0 use tga-image +>>>>16 ubyte 1 +>>>>>0 use tga-image +>>>>16 ubyte 8 +>>>>>0 use tga-image +>>>>16 ubyte 15 +>>>>>0 use tga-image +>>>>16 ubyte 16 +>>>>>0 use tga-image +>>>>16 ubyte 24 +>>>>>0 use tga-image +>>>>16 ubyte 32 +>>>>>0 use tga-image # display tga bitmap image information 0 name tga-image >2 ubyte <34 Targa image data @@ -174,43 +176,43 @@ 0 search/1 P1 >0 regex/4 P1[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, bitmap -!:strength + 45 +>>0 string x \b, bitmap +!:strength + 65 !:mime image/x-portable-bitmap 0 search/1 P2 >0 regex/4 P2[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, greymap -!:strength + 45 +>>0 string x \b, greymap +!:strength + 65 !:mime image/x-portable-greymap 0 search/1 P3 >0 regex/4 P3[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, pixmap -!:strength + 45 +>>0 string x \b, pixmap +!:strength + 65 !:mime image/x-portable-pixmap 0 string P4 >0 regex/4 P4[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, rawbits, bitmap -!:strength + 45 +>>0 string x \b, rawbits, bitmap +!:strength + 65 !:mime image/x-portable-bitmap 0 string P5 >0 regex/4 P5[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, rawbits, greymap -!:strength + 45 +>>0 string x \b, rawbits, greymap +!:strength + 65 !:mime image/x-portable-greymap 0 string P6 >0 regex/4 P6[\040\t\f\r\n] >>0 use netpbm ->>>0 string x \b, rawbits, pixmap -!:strength + 45 +>>0 string x \b, rawbits, pixmap +!:strength + 65 !:mime image/x-portable-pixmap 0 string P7 Netpbm PAM image file @@ -235,7 +237,7 @@ # Canon RAW version 1 (CRW) files are a type of Canon Image File Format # (CIFF) file. These are apparently all little-endian. # From: Adam Buchbinder <adam.buchbinder@gmail.com> -# URL: http://www.sno.phy.queensu.ca/~phil/exiftool/canon_raw.html +# URL: https://www.sno.phy.queensu.ca/~phil/exiftool/canon_raw.html 0 string II\x1a\0\0\0HEAPCCDR Canon CIFF raw image data !:mime image/x-canon-crw >16 leshort x \b, version %d. @@ -245,9 +247,10 @@ # number. Put this above the TIFF test to make sure we detect them. # These are apparently all little-endian. # From: Adam Buchbinder <adam.buchbinder@gmail.com> -# URL: http://libopenraw.freedesktop.org/wiki/Canon_CR2 +# URL: https://libopenraw.freedesktop.org/wiki/Canon_CR2 0 string II\x2a\0\x10\0\0\0CR Canon CR2 raw image data !:mime image/x-canon-cr2 +!:strength +80 >10 byte x \b, version %d. >11 byte x \b%d @@ -437,6 +440,7 @@ # Standard PNG image. 0 string \x89PNG\x0d\x0a\x1a\x0a\x00\x00\x00\x0DIHDR PNG image data !:mime image/png +!:ext png !:strength +10 >16 use png-ihdr @@ -444,6 +448,7 @@ 0 string \x89PNG\x0d\x0a\x1a\x0a\x00\x00\x00\x04CgBI >24 string \x00\x00\x00\x0DIHDR PNG image data (CgBI) !:mime image/png +!:ext png !:strength +10 >>32 use png-ihdr @@ -565,7 +570,7 @@ # at offset 8 starts imagedata followed by "RGB " marker # PC bitmaps (OS/2, Windows BMP files) (Greg Roelofs, newt@uchicago.edu) -# http://en.wikipedia.org/wiki/BMP_file_format#DIB_header_.\ +# https://en.wikipedia.org/wiki/BMP_file_format#DIB_header_.\ # 28bitmap_information_header.29 0 string BM >14 leshort 12 PC bitmap, OS/2 1.x format @@ -755,7 +760,7 @@ >5 byte 0xFF (black background) # From: Alex Myczko <alex@aiei.ch> -# http://www.atarimax.com/jindroush.atari.org/afmtatr.html +# https://www.atarimax.com/jindroush.atari.org/afmtatr.html 0 leshort 0x0296 Atari ATR image # XXX: @@ -781,8 +786,8 @@ # PCX image files # From: Dan Fandrich <dan@coneharvesters.com> -# updated by Joerg Jenderek at Feb 2013 by http://de.wikipedia.org/wiki/PCX -# http://web.archive.org/web/20100206055706/http://www.qzx.com/pc-gpe/pcx.txt +# updated by Joerg Jenderek at Feb 2013 by https://de.wikipedia.org/wiki/PCX +# https://web.archive.org/web/20100206055706/http://www.qzx.com/pc-gpe/pcx.txt # GRR: original test was still too general as it catches xbase examples T5.DBT,T6.DBT with 0xa000000 # test for bytes 0x0a,version byte (0,2,3,4,5),compression byte flag(0,1), bit depth (>0) of PCX or T5.DBT,T6.DBT 0 ubelong&0xffF8fe00 0x0a000000 @@ -849,7 +854,7 @@ # Update: Joerg Jenderek # See http://fileformats.archiveteam.org/wiki/GEM_Raster # For variations, also see: -# http://www.seasip.info/Gem/ff_img.html (Ventura) +# https://www.seasip.info/Gem/ff_img.html (Ventura) # http://www.atari-wiki.com/?title=IMG_file (XIMG, STTT) # http://www.fileformat.info/format/gemraster/spec/index.htm (XIMG, STTT) # http://sylvana.net/1stguide/1STGUIDE.ENG (TIMG) @@ -877,7 +882,7 @@ 0 name gem_info # version is 2 for some XIMG and 1 for all others >0 beshort <0x0003 GEM -# http://www.snowstone.org.uk/riscos/mimeman/mimemap.txt +# https://www.snowstone.org.uk/riscos/mimeman/mimemap.txt !:mime image/x-gem # header_size 24 25 27 59 779 words for colored bitmaps >>2 beshort >9 @@ -1020,8 +1025,8 @@ # and related image processing software used by biologists. # From: Vebjorn Ljosa <vebjorn@ljosa.com> # BOOL values are two-byte integers; use them to rule out false positives. -# http://web.archive.org/web/20050317223257/www.cs.ubc.ca/spider/ladic/text/biorad.txt -# Samples: http://www.loci.wisc.edu/software/sample-data +# https://web.archive.org/web/20050317223257/www.cs.ubc.ca/spider/ladic/text/biorad.txt +# Samples: https://www.loci.wisc.edu/software/sample-data 14 leshort <2 >62 leshort <2 >>54 leshort 12345 Bio-Rad .PIC Image File @@ -1052,7 +1057,7 @@ # Originally by Marc Espie # Modified by Robert Minsk <robertminsk at yahoo.com> -# http://www.openexr.com/openexrfilelayout.pdf +# https://www.openexr.com/openexrfilelayout.pdf 0 lelong 20000630 OpenEXR image data, !:mime image/x-exr >4 lelong&0x000000ff x version %d, @@ -1116,7 +1121,7 @@ >768 beshort 7 bottom to top/right to left # From: Tom Hilinski <tom.hilinski@comcast.net> -# http://www.unidata.ucar.edu/packages/netcdf/ +# https://www.unidata.ucar.edu/packages/netcdf/ 0 string CDF\001 NetCDF Data Format data #----------------------------------------------------------------------- @@ -1141,7 +1146,7 @@ # http://www.xara.com/ for Windows and as GPL application for Linux 0 string XARA\243\243 Xara graphics file -# http://www.cartesianinc.com/Tech/ +# https://www.cartesianinc.com/Tech/ 0 string CPC\262 Cartesian Perceptual Compression image !:mime image/x-cpi @@ -1210,8 +1215,8 @@ # extension is also used for # Sony SRF raw image (image/x-sony-srf) # SRF map -# Terragen Surface Map (http://www.planetside.co.uk/terragen) -# FileLocator Pro search criteria file (http://www.mythicsoft.com/filelocatorpro) +# Terragen Surface Map (https://www.planetside.co.uk/terragen) +# FileLocator Pro search criteria file (https://www.mythicsoft.com/filelocatorpro) !:ext srf #!:mime image/x-garmin-srf # version 1.00,2.00,2.10,2.40,2.50 @@ -1232,7 +1237,7 @@ 0 string Xcur X11 cursor # Type: Olympus ORF raw images. -# URL: http://libopenraw.freedesktop.org/wiki/Olympus_ORF +# URL: https://libopenraw.freedesktop.org/wiki/Olympus_ORF # From: Adam Buchbinder <adam.buchbinder@gmail.com> 0 string MMOR Olympus ORF raw image data, big-endian !:mime image/x-olympus-orf @@ -1253,7 +1258,7 @@ #!mime image/vnd.radiance # From: Adam Buchbinder <adam.buchbinder@gmail.com> -# URL: http://www.mpi-inf.mpg.de/resources/pfstools/pfs_format_spec.pdf +# URL: https://www.mpi-inf.mpg.de/resources/pfstools/pfs_format_spec.pdf # Used by the pfstools packages. The regex matches for the image size could # probably use some work. The MIME type is made up; if there's one in # actual common use, it should replace the one below. @@ -1263,7 +1268,7 @@ >>1 regex \ [0-9]{4} \bx%s # Type: Foveon X3F -# URL: http://www.photofo.com/downloads/x3f-raw-format.pdf +# URL: https://www.photofo.com/downloads/x3f-raw-format.pdf # From: Adam Buchbinder <adam.buchbinder@gmail.com> # Note that the MIME type isn't defined anywhere that I can find; if # there's a canonical type for this format, it should replace this one. @@ -1283,8 +1288,8 @@ # From: "Tano M. Fotang" <mfotang@quanteq.com> 0 string \x46\x4d\x52\x00 ISO/IEC 19794-2 Format Minutiae Record (FMR) -# doc: http://www.shikino.co.jp/eng/products/images/FLOWER.jpg.zip -# example: http://www.shikino.co.jp/eng/products/images/FLOWER.wdp.zip +# doc: https://www.shikino.co.jp/eng/products/images/FLOWER.jpg.zip +# example: https://www.shikino.co.jp/eng/products/images/FLOWER.wdp.zip 90 bequad 0x574D50484F544F00 JPEG-XR Image >98 byte&0x08 =0x08 \b, hard tiling >99 byte&0x80 =0x80 \b, tiling present @@ -1336,7 +1341,7 @@ # From: Johan van der Knijff <johan.vanderknijff@kb.nl> # # BPG (Better Portable Graphics) format -# http://bellard.org/bpg/ +# https://bellard.org/bpg/ # http://fileformats.archiveteam.org/wiki/BPG # 0 string \x42\x50\x47\xFB BPG (Better Portable Graphics) @@ -1399,7 +1404,7 @@ >12 ubelong x \b%d # Type: Microsoft DirectDraw Surface (common data) -# URL: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/DDSFileReference/ddsfileformat.asp +# URL: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/DDSFileReference/ddsfileformat.asp # From: Morten Hustveit <morten@debian.org> # Updated by: David Korth <gerbilsoft@gerbilsoft.com> 0 name ms-directdraw-surface @@ -1501,7 +1506,7 @@ >>>>0x64 ulequad 0xC00000003FF00000 \b, A2B10G10R10 # Type: Microsoft DirectDraw Surface -# URL: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/DDSFileReference/ddsfileformat.asp +# URL: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/DDSFileReference/ddsfileformat.asp # From: Morten Hustveit <morten@debian.org> # Updated by: David Korth <gerbilsoft@gerbilsoft.com> 0 string/b DDS\040\174\000\000\000 Microsoft DirectDraw Surface (DDS): @@ -1510,7 +1515,7 @@ # Type: Sega PVR image. # From: David Korth <gerbilsoft@gerbilsoft.com> # References: -# - http://fabiensanglard.net/Mykaruga/tools/segaPVRFormat.txt +# - https://fabiensanglard.net/Mykaruga/tools/segaPVRFormat.txt # - https://github.com/yazgoo/pvrx2png # - https://github.com/nickworonekin/puyotools @@ -1553,7 +1558,7 @@ 0 string GBIX >0x10 string PVRT >>0x10 string DDS\040\174\000\000\000 Sega PVR (Xbox) image: ->>0x20 use ms-directdraw-surface +>>>0x20 use ms-directdraw-surface >>0x10 belong !0x44445320 Sega PVR image: >>>0x10 use sega-pvr-image-header >>0x08 lelong x \b, global index = %u @@ -1703,7 +1708,7 @@ >12 lelong 0x04030201 (little-endian) >>16 use khronos-ktx-endian-header >12 belong 0x04030201 (big-endian) ->>16 use ^khronos-ktx-endian-header +>>16 use \^khronos-ktx-endian-header # Type: Valve VTF texture. # From: David Korth <gerbilsoft@gerbilsoft.com> @@ -1793,3 +1798,165 @@ 0 string LinS Microsoft Paint image data (version 2.0) >4 leshort x (%d >>6 leshort x x %d) + +# reMarkable tablet internal file format (https://www.remarkable.com/) +# https://github.com/ax3l/lines-are-beautiful +# https://plasma.ninja/blog/devices/remarkable/binary/format/2017/12/26/\ +# reMarkable-lines-file-format.html#what-to-do-next +# from Axel Huebl +0 string reMarkable +>11 string lines +>>17 string with +>>>22 string selections +>>>>33 string and +>>>>>37 string layers +>>>>>>43 lelong x reMarkable tablet notebook lines, 1404 x 1872, %x page(s) + +# newer per-page files for the reMarkable +0 string reMarkable +>11 string .lines +>>18 string file, +>>>24 string version= +>>>>32 byte x reMarkable tablet page (v%c), 1404 x 1872, +>>>>>43 lelong x %d layer(s) + +# Type: PVR3 texture. +# From: David Korth <gerbilsoft@gerbilsoft.com> +# References: +# - http://cdn.imgtec.com/sdk-documentation/PVR+File+Format.Specification.pdf + +# PVR3 pixel formats. +0 name pvr3-pixel-format +>4 ulelong 0 +>>0 ulelong 0 PVRTC 2bpp RGB +>>0 ulelong 1 PVRTC 2bpp RGBA +>>0 ulelong 2 PVRTC 4bpp RGB +>>0 ulelong 3 PVRTC 4bpp RGBA +>>0 ulelong 4 PVRTC-II 2bpp +>>0 ulelong 5 PVRTC-II 4bpp +>>0 ulelong 6 ETC1 +>>0 ulelong 7 DXT1 +>>0 ulelong 8 DXT2 +>>0 ulelong 9 DXT3 +>>0 ulelong 10 DXT4 +>>0 ulelong 11 DXT5 +>>0 ulelong 12 BC4 +>>0 ulelong 13 BC5 +>>0 ulelong 14 BC6 +>>0 ulelong 15 BC7 +>>0 ulelong 16 UYVY +>>0 ulelong 17 YUY2 +>>0 ulelong 18 BW1bpp +>>0 ulelong 19 R9G9B9E5 Shared Exponent +>>0 ulelong 20 RGBG8888 +>>0 ulelong 21 GRGB8888 +>>0 ulelong 22 ETC2 RGB +>>0 ulelong 23 ETC2 RGBA +>>0 ulelong 24 ETC2 RGB A1 +>>0 ulelong 25 EAC R11 +>>0 ulelong 26 EAC RG11 +>>0 ulelong 27 ASTC_4x4 +>>0 ulelong 28 ASTC_5x4 +>>0 ulelong 29 ASTC_5x5 +>>0 ulelong 30 ASTC_6x5 +>>0 ulelong 31 ASTC_6x6 +>>0 ulelong 32 ASTC_8x5 +>>0 ulelong 33 ASTC_8x6 +>>0 ulelong 34 ASTC_8x8 +>>0 ulelong 35 ASTC_10x5 +>>0 ulelong 36 ASTC_10x6 +>>0 ulelong 37 ASTC_10x8 +>>0 ulelong 38 ASTC_10x10 +>>0 ulelong 39 ASTC_12x10 +>>0 ulelong 40 ASTC_12x12 +>>0 ulelong 41 ASTC_3x3x3 +>>0 ulelong 42 ASTC_4x3x3 +>>0 ulelong 43 ASTC_4x4x3 +>>0 ulelong 44 ASTC_4x4x4 +>>0 ulelong 45 ASTC_5x4x4 +>>0 ulelong 46 ASTC_5x5x4 +>>0 ulelong 47 ASTC_5x5x5 +>>0 ulelong 48 ASTC_6x5x5 +>>0 ulelong 49 ASTC_6x6x5 +>>0 ulelong 50 ASTC_6x6x6 +>4 ulelong !0 +>>0 byte !0 %c +>>1 byte !0 \b%c +>>2 byte !0 \b%c +>>3 byte !0 \b%c + +0 string PVR\x03 PVR 3.0 texture: +>0x18 ulelong x %u x +>0x1C ulelong x %u +>0x20 ulelong >1 x %u +>0x08 byte x \b, +>0x08 use pvr3-pixel-format +>0x10 ulelong 1 \b, sRGB +>0x04 ulelong&0x02 0x02 \b, premultiplied alpha + +# Type: Microsoft Xbox XPR0 texture. +# From: David Korth <gerbilsoft@gerbilsoft.com> +# References: +# - https://github.com/Cxbx-Reloaded/Cxbx-Reloaded/blob/develop/src/core/hle/D3D8/XbD3D8Types.h + +# XPR pixel formats. +0 name xbox-xpr-pixel-format +>0 byte 0x00 L8 +>0 byte 0x01 AL8 +>0 byte 0x02 ARGB1555 +>0 byte 0x03 RGB555 +>0 byte 0x04 ARGB4444 +>0 byte 0x05 RGB565 +>0 byte 0x06 ARGB8888 +>0 byte 0x07 xRGB8888 +>0 byte 0x0B P8 +>0 byte 0x0C DXT1 +>0 byte 0x0E DXT2 +>0 byte 0x0F DXT4 +>0 byte 0x10 Linear ARGB1555 +>0 byte 0x11 Linear RGB565 +>0 byte 0x12 Linear ARGB8888 +>0 byte 0x13 Linear L8 +>0 byte 0x16 Linear R8B8 +>0 byte 0x17 Linear G8B8 +>0 byte 0x19 A8 +>0 byte 0x1A A8L8 +>0 byte 0x1B Linear AL8 +>0 byte 0x1C Linear RGB555 +>0 byte 0x1D Linear ARGB4444 +>0 byte 0x1E Linear xRGB8888 +>0 byte 0x1F Linear A8 +>0 byte 0x20 Linear A8L8 +>0 byte 0x24 YUY2 +>0 byte 0x25 UYVY +>0 byte 0x27 L6V5U5 +>0 byte 0x28 V8U8 +>0 byte 0x29 R8B8 +>0 byte 0x2A D24S8 +>0 byte 0x2B F24S8 +>0 byte 0x2C D16 +>0 byte 0x2D F16 +>0 byte 0x2E Linear D24S8 +>0 byte 0x2F Linear F24S8 +>0 byte 0x30 Linear D16 +>0 byte 0x31 Linear F16 +>0 byte 0x32 L16 +>0 byte 0x33 V16U16 +>0 byte 0x35 Linear L16 +>0 byte 0x36 Linear V16U16 +>0 byte 0x37 Linear L6V5U5 +>0 byte 0x38 RGBA5551 +>0 byte 0x39 RGBA4444 +>0 byte 0x3A QWVU8888 +>0 byte 0x3B BGRA8888 +>0 byte 0x3C RGBA8888 +>0 byte 0x3D Linear RGBA5551 +>0 byte 0x3E Linear RGBA4444 +>0 byte 0x3F Linear ABGR8888 +>0 byte 0x40 Linear BGRA8888 +>0 byte 0x41 Linear RGBA8888 +>0 byte 0x64 Vertex Data + +0 string XPR0 Microsoft Xbox XPR0 texture +>0x19 byte x \b, format: +>>0x19 use xbox-xpr-pixel-format diff --git a/magic/Magdir/intel b/magic/Magdir/intel index 8a7eae4ee1ee..85e2abef457d 100644 --- a/magic/Magdir/intel +++ b/magic/Magdir/intel @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: intel,v 1.16 2017/11/14 15:48:36 christos Exp $ +# $File: intel,v 1.17 2018/08/01 10:34:03 christos Exp $ # intel: file(1) magic for x86 Unix # # Various flavors of x86 UNIX executable/object (other than Xenix, which @@ -42,6 +42,10 @@ #>12 lelong >0 not stripped # no hint found, that at offset 22 is version #>22 leshort >0 - version %d +0 leshort 0x0200 +>0 use display-coff +0 leshort 0x8664 +>0 use display-coff # rom: file(1) magic for BIOS ROM Extensions found in intel machines # mapped into memory between 0xC0000 and 0xFFFFF diff --git a/magic/Magdir/isz b/magic/Magdir/isz index af68db2b3e2e..4d9c03084445 100644 --- a/magic/Magdir/isz +++ b/magic/Magdir/isz @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: isz,v 1.4 2017/03/17 21:35:28 christos Exp $ +# $File: isz,v 1.5 2019/04/19 00:42:27 christos Exp $ # ISO Zipped file format -# http://www.ezbsystems.com/isz/iszspec.txt +# https://www.ezbsystems.com/isz/iszspec.txt 0 string IsZ! ISO Zipped file >4 byte x \b, header size %u >5 byte x \b, version %u diff --git a/magic/Magdir/java b/magic/Magdir/java index 21acf295e086..b9854e54c159 100644 --- a/magic/Magdir/java +++ b/magic/Magdir/java @@ -1,6 +1,6 @@ #------------------------------------------------------------ -# $File: java,v 1.18 2015/11/29 22:08:14 christos Exp $ +# $File: java,v 1.21 2019/02/18 17:58:50 christos Exp $ # Java ByteCode and Mach-O binaries (e.g., Mac OS X) use the # same magic number, 0xcafebabe, so they are both handled # in the entry called "cafebabe". @@ -22,6 +22,24 @@ # Java HPROF dumps # https://java.net/downloads/heap-snapshot/hprof-binary-format.html 0 string JAVA\x20PROFILE\x201.0. ->0x12 short 0 ->>0x11 ushort-0x31 <2 Java HPROF dump, ->>0x17 beqdate/1000 x created %s +>0x12 byte 0 +>>0x11 ubyte-0x31 <2 Java HPROF dump, +>>>0x17 beqdate/1000 x created %s + +# Java jmod module +# See https://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.base/share/classes/jdk/internal/jmod/JmodFile.java +# Grr. 2 byte magic "JM", really? In 2019? +0 belong 0x4a4d0100 Java jmod module version 1.0 +!:mime application/x-java-jmod + +# Java jlinked image +# See https://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.base/share/native/libjimage/imageFile.hpp +0 belong 0xcafedada Java module image (big endian) +>4 beshort >0x00 \b, version %d +>6 beshort x \b.%d +!:mime application/x-java-image + +0 lelong 0xcafedada Java module image (little endian) +>6 leshort >0x00 \b, version %d +>4 leshort x \b.%d +!:mime application/x-java-image diff --git a/magic/Magdir/jpeg b/magic/Magdir/jpeg index 57adb1e546b0..52c9ad36203e 100644 --- a/magic/Magdir/jpeg +++ b/magic/Magdir/jpeg @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: jpeg,v 1.31 2017/03/17 21:35:28 christos Exp $ +# $File: jpeg,v 1.32 2018/10/01 18:58:29 christos Exp $ # JPEG images # SunOS 5.5.1 had # @@ -51,21 +51,21 @@ >>4 byte x \b, baseline, precision %d >>7 beshort x \b, %dx >>5 beshort x \b%d ->>9 byte x \b, frames %d +>>9 byte x \b, components %d >0 beshort 0xFFC1 >>(2.S+2) use jpeg_segment >>4 byte x \b, extended sequential, precision %d >>7 beshort x \b, %dx >>5 beshort x \b%d ->>9 byte x \b, frames %d +>>9 byte x \b, components %d >0 beshort 0xFFC2 >>(2.S+2) use jpeg_segment >>4 byte x \b, progressive, precision %d >>7 beshort x \b, %dx >>5 beshort x \b%d ->>9 byte x \b, frames %d +>>9 byte x \b, components %d # Define Huffman Tables >0 beshort 0xFFC4 diff --git a/magic/Magdir/keepass b/magic/Magdir/keepass index ba132d57de83..3d26efa5c56f 100644 --- a/magic/Magdir/keepass +++ b/magic/Magdir/keepass @@ -1,12 +1,12 @@ #------------------------------------------------------------------------------ -# $File: keepass,v 1.1 2012/12/24 22:14:56 christos Exp $ +# $File: keepass,v 1.2 2019/04/19 00:42:27 christos Exp $ # keepass: file(1) magic for KeePass file # # Keepass Password Safe: -# * original one: http://keepass.info/ -# * *nix port: http://www.keepassx.org/ -# * android port: http://code.google.com/p/keepassdroid/ +# * original one: https://keepass.info/ +# * *nix port: https://www.keepassx.org/ +# * android port: https://code.google.com/p/keepassdroid/ 0 lelong 0x9AA2D903 Keepass password database >4 lelong 0xB54BFB65 1.x KDB diff --git a/magic/Magdir/kerberos b/magic/Magdir/kerberos index 93b8f79de99b..df6dc5236446 100644 --- a/magic/Magdir/kerberos +++ b/magic/Magdir/kerberos @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: kerberos,v 1.2 2017/03/17 21:35:28 christos Exp $ +# $File: kerberos,v 1.3 2019/04/19 00:42:27 christos Exp $ # kerberos: MIT kerberos file binary formats # @@ -24,7 +24,7 @@ # can refer to ${name} # - Provide a way to format strings as hex values # -# http://www.gnu.org/software/shishi/manual/html_node/\ +# https://www.gnu.org/software/shishi/manual/html_node/\ # The-Keytab-Binary-File-Format.html # diff --git a/magic/Magdir/kicad b/magic/Magdir/kicad new file mode 100644 index 000000000000..00c9ead5ae9b --- /dev/null +++ b/magic/Magdir/kicad @@ -0,0 +1,69 @@ + +#------------------------------------------------------------------------------ +# $File: kicad,v 1.1 2018/10/01 18:39:21 christos Exp $ +# kicad: file(1) magic for KiCad files +# +# See +# +# http://kicad-pcb.org +# + +# KiCad Schematic Document +0 string EESchema +>8 byte 0x20 +>>9 string Schematic +>>>18 byte 0x20 KiCad Schematic Document +!:ext sch/bak +>>>>24 string Version +>>>>>31 byte 0x20 +>>>>>>32 string x (Version %s) + +# KiCad Symbol Library +0 string EESchema-LIBRARY +>16 byte 0x20 KiCad Symbol Library +!:ext lib +>>17 string Version +>>>24 byte 0x20 +>>>>25 string x (Version %s) + +# KiCad Symbol Library Documentation +0 string EESchema-DOCLIB +>15 byte 0x20 KiCad Symbol Library Documentation +!:ext dcm +>>17 string Version +>>>24 byte 0x20 +>>>>25 string x (Version %s) + +# KiCad Board Layout +0 string (kicad_pcb +>10 byte 0x20 KiCad Board Layout +!:ext kicad_pcb/kicad_pcb-bak +>>11 string (version +>>>19 byte 0x20 +>>>>20 byte x (Version %c) + +# KiCad Footprint +0 string (module +>7 byte 0x20 KiCad Footprint +!:ext kicad_mod + +# KiCad Footprint (Legacy) +0 string PCBNEW-LibModule-V1 KiCad Footprint (Legacy) +!:ext mod + +# KiCad Netlist +0 string (export +>7 byte 0x20 KiCad Netlist +!:ext net + +# KiCad Symbol Library Table +0 string (sym_lib_table +>14 byte 0xA KiCad Symbol Library Table +>14 byte 0xD KiCad Symbol Library Table +>14 byte 0x20 KiCad Symbol Library Table + +# KiCad Footprint Library Table +0 string (fp_lib_table +>13 byte 0xA KiCad Footprint Library Table +>13 byte 0xD KiCad Footprint Library Table +>13 byte 0x20 KiCad Footprint Library Table diff --git a/magic/Magdir/kml b/magic/Magdir/kml index d603ea806288..dcdf64cf454f 100644 --- a/magic/Magdir/kml +++ b/magic/Magdir/kml @@ -1,14 +1,14 @@ #------------------------------------------------------------------------------ -# $File: kml,v 1.4 2017/03/17 21:35:28 christos Exp $ +# $File: kml,v 1.5 2019/04/19 00:42:27 christos Exp $ # Type: Google KML, formerly Keyhole Markup Language # Future development of this format has been handed # over to the Open Geospatial Consortium. -# http://www.opengeospatial.org/standards/kml/ +# https://www.opengeospatial.org/standards/kml/ # From: Asbjoern Sloth Toennesen <asbjorn@lila.io> 0 string/t \<?xml >20 search/400 \ xmlns= ->>&0 regex ['"]http://earth.google.com/kml Google KML document +>>&0 regex ['"]https://earth.google.com/kml Google KML document !:mime application/vnd.google-earth.kml+xml >>>&1 string 2.0' \b, version 2.0 >>>&1 string 2.1' \b, version 2.1 @@ -18,15 +18,15 @@ # Type: OpenGIS KML, formerly Keyhole Markup Language # This standard is maintained by the # Open Geospatial Consortium. -# http://www.opengeospatial.org/standards/kml/ +# https://www.opengeospatial.org/standards/kml/ # From: Asbjoern Sloth Toennesen <asbjorn@lila.io> ->>&0 regex ['"]http://www.opengis.net/kml OpenGIS KML document +>>&0 regex ['"]https://www.opengis.net/kml OpenGIS KML document !:mime application/vnd.google-earth.kml+xml >>>&1 string/t 2.2 \b, version 2.2 #------------------------------------------------------------------------------ # Type: Google KML Archive (ZIP based) -# http://code.google.com/apis/kml/documentation/kml_tut.html +# https://code.google.com/apis/kml/documentation/kml_tut.html # From: Asbjoern Sloth Toennesen <asbjorn@lila.io> 0 string PK\003\004 >4 byte 0x14 diff --git a/magic/Magdir/linux b/magic/Magdir/linux index 0486656f6a6b..ed7dcd10a251 100644 --- a/magic/Magdir/linux +++ b/magic/Magdir/linux @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: linux,v 1.65 2018/07/16 12:32:08 christos Exp $ +# $File: linux,v 1.67 2019/04/19 00:42:27 christos Exp $ # linux: file(1) magic for Linux files # # Values for Linux/i386 binaries, from Daniel Quinlan <quinlan@yggdrasil.com> @@ -56,7 +56,7 @@ # # PSF fonts, from H. Peter Anvin <hpa@yggdrasil.com> # Updated by Adam Buchbinder <adam.buchbinder@gmail.com> -# See: http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html +# See: https://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html 0 leshort 0x0436 Linux/i386 PC Screen Font v1 data, >2 byte&0x01 0 256 characters, >2 byte&0x01 !0 512 characters, @@ -79,7 +79,7 @@ 4076 string SWAPSPACE2LINHIB0001 Linux/i386 swap file (new style) (compressed hibernate) # according to man page of mkswap (8) March 1999 # volume label and UUID Russell Coker -# http://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ +# https://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ 4086 string SWAPSPACE2 Linux/i386 swap file (new style), >0x400 long x version %d (4K pages), >0x404 long x size %d pages, @@ -111,8 +111,13 @@ # and Nicolas Lichtmaier <nick@debian.org> # All known start with: b8 c0 07 8e d8 b8 00 90 8e c0 b9 00 01 29 f6 29 # Linux kernel boot images (i386 arch) (Wolfram Kleff) +# URL: https://www.kernel.org/doc/Documentation/x86/boot.txt 514 string HdrS Linux kernel !:strength + 55 +# often no extension like in linux, vmlinuz, bzimage or memdisk but sometimes +# Acronis Recovery kernel64.dat and Plop Boot Manager plpbtrom.bin +# DamnSmallLinux 1.5 damnsmll.lnx +!:ext /dat/bin/lnx >510 leshort 0xAA55 x86 boot executable >>518 leshort >0x1ff >>>529 byte 0 zImage, @@ -203,7 +208,10 @@ # Linux ARM compressed kernel image # From: Kevin Cernekee <cernekee@gmail.com> +# Update: Joerg Jenderek 36 lelong 0x016f2818 Linux kernel ARM boot executable zImage (little-endian) +# raspian "kernel7.img", Vu+ Ultimo4K "kernel_auto.bin" +!:ext img/bin 36 belong 0x016f2818 Linux kernel ARM boot executable zImage (big-endian) ############################################################################ @@ -232,7 +240,7 @@ # 0 lelong&0xFF00FFFF 0x17000301 ld86 SPARC executable # SYSLINUX boot logo files (from 'ppmtolss16' sources) -# http://www.syslinux.org/wiki/index.php/SYSLINUX#Display_graphic_from_filename: +# https://www.syslinux.org/wiki/index.php/SYSLINUX#Display_graphic_from_filename: # file extension .lss .16 0 lelong =0x1413f33d SYSLINUX' LSS16 image data # syslinux-4.05/mime/image/x-lss16.xml @@ -361,7 +369,7 @@ >>&1 string x %s...) # Systemd journald files -# See http://www.freedesktop.org/wiki/Software/systemd/journal-files/. +# See https://www.freedesktop.org/wiki/Software/systemd/journal-files/. # From: Zbigniew Jedrzejewski-Szmek <zbyszek@in.waw.pl> # check magic @@ -453,7 +461,7 @@ # the default version of locate on Arch Linux (and others). # File path: /var/lib/mlocate/mlocate.db by default (but configurable) # Site: https://fedorahosted.org/mlocate/ -# Format docs: http://linux.die.net/man/5/mlocate.db +# Format docs: https://linux.die.net/man/5/mlocate.db # Type: mlocate database file # URL: https://fedorahosted.org/mlocate/ # From: Wander Nauta <info@wandernauta.nl> @@ -469,14 +477,14 @@ 0 lelong 0x47361222 iproute2 addresses dump # Image and service files for CRIU tool. -# URL: http://criu.org +# URL: https://criu.org # From: Pavel Emelyanov <xemul@parallels.com> 0 lelong 0x54564319 CRIU image file v1.1 0 lelong 0x55105940 CRIU service file 0 lelong 0x58313116 CRIU inventory # Kdump compressed dump files -# http://sourceforge.net/p/makedumpfile/code/ci/master/tree/IMPLEMENTATION +# https://sourceforge.net/p/makedumpfile/code/ci/master/tree/IMPLEMENTATION 0 string KDUMP Kdump compressed dump >8 long x v%d diff --git a/magic/Magdir/lisp b/magic/Magdir/lisp index 43d102e1dd76..d32cc101a90a 100644 --- a/magic/Magdir/lisp +++ b/magic/Magdir/lisp @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: lisp,v 1.25 2017/03/17 21:35:28 christos Exp $ +# $File: lisp,v 1.26 2019/04/19 00:42:27 christos Exp $ # lisp: file(1) magic for lisp programs # # various lisp types, from Daniel Quinlan (quinlan@yggdrasil.com) @@ -27,7 +27,7 @@ !:mime text/x-lisp # URL: https://en.wikipedia.org/wiki/Emacs_Lisp -# Reference: http://ftp.gnu.org/old-gnu/emacs/elisp-manual-18-1.03.tar.gz +# Reference: https://ftp.gnu.org/old-gnu/emacs/elisp-manual-18-1.03.tar.gz # Update: Joerg Jenderek # Emacs 18 - this is always correct, but not very magical. 0 string \012( diff --git a/magic/Magdir/llvm b/magic/Magdir/llvm index 1f15eac7b52e..2691ef1ac92f 100644 --- a/magic/Magdir/llvm +++ b/magic/Magdir/llvm @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: llvm,v 1.8 2013/01/12 03:09:51 christos Exp $ +# $File: llvm,v 1.9 2019/04/19 00:42:27 christos Exp $ # llvm: file(1) magic for LLVM byte-codes -# URL: http://llvm.org/docs/BitCodeFormat.html +# URL: https://llvm.org/docs/BitCodeFormat.html # From: Al Stone <ahs3@fc.hp.com> 0 string llvm LLVM byte-codes, uncompressed diff --git a/magic/Magdir/lua b/magic/Magdir/lua index 4c578be60ff5..0e47c2f9122a 100644 --- a/magic/Magdir/lua +++ b/magic/Magdir/lua @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: lua,v 1.6 2013/01/09 16:23:17 christos Exp $ +# $File: lua,v 1.7 2019/04/19 00:42:27 christos Exp $ # lua: file(1) magic for Lua scripting language -# URL: http://www.lua.org/ +# URL: https://www.lua.org/ # From: Reuben Thomas <rrt@sc3d.org>, Seo Sanghyeon <tinuviel@sparcs.kaist.ac.kr> # Lua scripts diff --git a/magic/Magdir/m4 b/magic/Magdir/m4 index 3a1c6d19f7db..587ebe80c66d 100644 --- a/magic/Magdir/m4 +++ b/magic/Magdir/m4 @@ -1,9 +1,11 @@ #------------------------------------------------------------------------------ -# $File: m4,v 1.2 2017/08/14 07:40:38 christos Exp $ +# $File: m4,v 1.3 2019/02/27 16:46:23 christos Exp $ # make: file(1) magic for M4 scripts # -0 regex \^dnl\ M4 macro processor script text +0 search/8192 dnl +>0 regex \^dnl\ M4 macro processor script text !:mime text/x-m4 -0 regex \^AC_DEFUN\\(\\[ M4 macro processor script text +0 search/8192 AC_DEFUN +>0 regex \^AC_DEFUN\\(\\[ M4 macro processor script text !:strength + 15 !:mime text/x-m4 diff --git a/magic/Magdir/macintosh b/magic/Magdir/macintosh index 249a4d8050b6..2a9f7a7b9897 100644 --- a/magic/Magdir/macintosh +++ b/magic/Magdir/macintosh @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: macintosh,v 1.28 2017/12/05 02:17:48 christos Exp $ +# $File: macintosh,v 1.29 2019/04/19 00:42:27 christos Exp $ # macintosh description # # BinHex is the Macintosh ASCII-encoded file format (see also "apple") @@ -62,7 +62,7 @@ # MacBinary format (Eric Fischer, enf@pobox.com) # Update: Joerg Jenderek # URL: https://en.wikipedia.org/wiki/MacBinary -# Reference: http://files.stairways.com/other/macbinaryii-standard-info.txt +# Reference: https://files.stairways.com/other/macbinaryii-standard-info.txt # # Unfortunately MacBinary doesn't really have a magic number prior # to the MacBinary III format. @@ -180,8 +180,8 @@ # Apple Type/Creator Database # URL: https://en.wikipedia.org/wiki/Type_code -# Reference: http://www.lacikam.co.il/tcdb/ -# http://www.macdisk.com/macsigen.php +# Reference: https://www.lacikam.co.il/tcdb/ +# https://www.macdisk.com/macsigen.php # Note: classic Mac OS files have two 4 character codes for type and creator. # Thereby the Finder attach documents types to applications. @@ -379,7 +379,7 @@ >0x424 pstring x volume name: %s # *.hfs updated by Joerg Jenderek -# http://en.wikipedia.org/wiki/Hierarchical_File_System +# https://en.wikipedia.org/wiki/Hierarchical_File_System # "BD" gives many false positives 0x400 beshort 0x4244 # ftp://ftp.mars.org/pub/hfs/hfsutils-3.2.6.tar.gz/hfsutils-3.2.6/libhfs/apple.h @@ -390,7 +390,7 @@ !:mime application/x-apple-diskimage #!:apple hfsdINIT #!:apple MACSdisk -# http://www.macdisk.com/macsigen.php +# https://www.macdisk.com/macsigen.php #!:apple ddskdevi !:apple ????devi # https://en.wikipedia.org/wiki/Apple_Disk_Image @@ -402,7 +402,7 @@ >>>0x40a beshort &0x0200 (spared blocks) >>>0x40a beshort &0x0800 (unclean) >>>0x47C beshort 0x482B (Embedded HFS+ Volume) -# http://www.epochconverter.com/ +# https://www.epochconverter.com/ # 0x7C245F00 seconds ~ 2082758400 ~ 01 Jan 2036 00:00:00 ~ 66 years to 1970 # 0x7C25B080 seconds ~ 2082844800 ~ 02 Jan 2036 00:00:00 # construct not working @@ -458,7 +458,7 @@ 0 string BOMStore Mac OS X bill of materials (BOM) file # From: Adam Buchbinder <adam.buchbinder@gmail.com> -# URL: http://en.wikipedia.org/wiki/Datafork_TrueType +# URL: https://en.wikipedia.org/wiki/Datafork_TrueType # Derived from the 'fondu' and 'ufond' source code (fondu.sf.net). 'sfnt' is # TrueType; 'POST' is PostScript. 'FONT' and 'NFNT' sometimes appear, but I # don't know what they mean. diff --git a/magic/Magdir/mail.news b/magic/Magdir/mail.news index a61bc72cb8d9..c58b3710af28 100644 --- a/magic/Magdir/mail.news +++ b/magic/Magdir/mail.news @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: mail.news,v 1.23 2015/06/29 14:44:26 christos Exp $ +# $File: mail.news,v 1.24 2019/04/19 00:42:27 christos Exp $ # mail.news: file(1) magic for mail and news # # Unfortunately, saved netnews also has From line added in some news software. @@ -57,7 +57,7 @@ #0 string \<!--\ MHonArc text/html; x-type=mhonarc # Cyrus: file(1) magic for compiled Cyrus sieve scripts -# URL: http://www.cyrusimap.org/docs/cyrus-imapd/2.4.6/internal/bytecode.php +# URL: https://www.cyrusimap.org/docs/cyrus-imapd/2.4.6/internal/bytecode.php # URL: http://git.cyrusimap.org/cyrus-imapd/tree/sieve/bytecode.h?h=master # From: Philipp Hahn <hahn@univention.de> diff --git a/magic/Magdir/map b/magic/Magdir/map index e02b2052e225..af5f24ef02b4 100644 --- a/magic/Magdir/map +++ b/magic/Magdir/map @@ -1,16 +1,16 @@ #------------------------------------------------------------------------------ -# $File: map,v 1.4 2015/08/10 05:18:27 christos Exp $ +# $File: map,v 1.7 2019/04/30 04:02:04 christos Exp $ # map: file(1) magic for Map data # -# Garmin .FIT files http://pub.ks-and-ks.ne.jp/cycling/edge500_fit.shtml +# Garmin .FIT files https://pub.ks-and-ks.ne.jp/cycling/edge500_fit.shtml 8 string .FIT FIT Map data >15 byte 0 >>35 belong x \b, unit id %d >>39 lelong x \b, serial %u -# http://pub.ks-and-ks.ne.jp/cycling/edge500_fit.shtml +# https://pub.ks-and-ks.ne.jp/cycling/edge500_fit.shtml # 20 years after unix epoch # TZ=GMT date -d '1989-12-31 0:00' +%s >>43 leldate+631065600 x \b, %s @@ -26,8 +26,298 @@ >>53 byte 8 \b (Elevations) >>53 byte 10 \b (Totals) +# Summary: Garmin map +# From: Joerg Jenderek +# URL: https://en.wikipedia.org/wiki/Garmin_.img +# Reference: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/IMG_File_Format +# sourceforge.net/projects/garmin-img/files/IMG%20File%20Format/1.0/imgformat-1.0.pdf +# GRR: similar to MBR boot sector handled by ./filesystems +0x1FE leshort =0xAA55 +# look for valid map signature +>0x13 string =IMG\0 +>>0 use garmin-map +0 name garmin-map +>0 ubyte x Garmin +!:mime application/x-garmin-map +# If non-zero, every byte of the entire .img file is to be XORed with this value +>0 ubyte !0 \b, 0x%x XORed +# goto block before FAT +>(0x40.b*512) ubyte x +# 1st fat name "DLLINFO TXT" only found for vpm +>>&512 string =DLLINFO\ TXT map (Voice Processing) +# there exist 2 other Garmin VPM formats; see ./audio +!:ext vpm +# Deutsch__Yannick_D4481-00_0210.vpm +#>>>512 search/0x0116da60/s RIFF \b; with +# determine type voice type by ./riff +#>>>>&0 indirect x \b +>>&512 string !DLLINFO\ TXT map +!:ext img +# 9 zeros +>1 ubelong !0 \b, zeroes 0x%x +# Map's version major +>8 ubyte x v%u +# Map's version minor +>9 ubyte x \b.%.2u +# Map description[20], 0x20 padded +>0x49 string x %.20s +# Map name, continued (0x20 padded, \0 terminated) +>0x65 string >\ \b%.31s +# Update year (+1900 for val >= 0x63, +2000 for val <= 0x62) +>0xB ubyte x \b, updated +>>0xB ubyte >0x62 +>>>0xB ubyte-100 x 20%.2u +>>0xB ubyte <0x63 +>>>0xB ubyte x 20%.2u +# Update month (0-11) +>0xA ubyte x \b-%.2u +# All zeroes +>0xc uleshort !0 \b, zeroes 0x%x +# Mapsource flag, 1 - file created by Mapsource, 0 - Garmin map visible in Basecamp and Homeport +#>0xE ubyte !0 \b, Mapsource flag 0x%x +>0xE ubyte 1 \b, Mapsource +# Checksum, sum of all bytes modulo 256 should be 0 +#>0xF ubyte x \b, Checksum 0x%x +# Signature: DSKIMG 0x00 or DSDIMG 0x00 for demo map +>0x10 string !DSKIMG \b, signature "%.7s" +>0x39 use garmin-date +# Map file identifier like GARMIN\0 +>0x41 string !GARMIN \b, id "%.7s" +# Block size exponent, E1; appears to always be 0x09; minimum block size 512 bytes +>0x61 ubyte !0x09 \b, E1=%u +# Block size exponent, E2 ; file blocksize=2**(E1+E2) +>>0x62 ubyte x \b, E2=%u +>0x61 ubyte =0x09 \b, blocksize +>>0x62 ubyte 0 512 +>>0x62 ubyte 1 1024 +>>0x62 ubyte 2 2048 +>>0x62 ubyte 3 4096 +>>0x62 ubyte 4 8192 +>>0x62 ubyte 5 16384 +>>0x62 default x +>>>0x62 ubyte x E2=%u +# MBR signature +>0x1FE leshort !0xAA55 \b, invalid MBR +# 512 zeros +>0x200 uquad !0 \b, zeroes 0x%llx +# First sub-file offset (absolute); sometimes NO/UNKNOWN sub file! +>0x40C ulelong >0 \b, at 0x%x +# sub-file Header length +#>>(0x40C.l) uleshort x \b, header len 0x%x +>>(0x40C.l) uleshort x %u bytes +# sub-file Type[10] like "GARMIN RGN" "GARMIN TRE", "GARMIN TYP", etc. +>>(0x40C.l+2) ubyte >0x1F +>>>(0x40C.l+2) ubyte <0xFF +>>>>(0x40C.l+2) string x "%.10s" +# 0x00 for most maps, 0x80 for locked maps (City Nav, City Select, etc.) +>>>>(0x40C.l+13) ubyte >0 \b, locked 0x%x +# Block sequence numbers like 0000 0100 0200 ... FFFF +# >0x420 ubequad >0 \b, seq. 0x%16.16llx +# >>0x428 ubequad >0 \b%16.16llx +# >>>0x430 ubequad >0 \b%16.16llx +# >>>>0x438 ubequad >0 \b%16.16llx +# >>>>>0x440 ubequad >0 \b%16.16llx +# >>>>>>0x448 ubequad >0 \b%16.16llx +# >>>>>>>0x450 ubequad >0 \b%16.16llx +# >>>>>>>>0x458 ubequad >0 \b%16.16llx +# >>>>>>>>>0x460 ubequad >0 \b%16.16llx +# >>>>>>>>>>0x468 ubequad >0 \b%16.16llx +# >>>>>>>>>>>0x470 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>0x478 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>0x480 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>>0x488 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>>>0x490 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>>>>0x498 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>>>>>0x4A0 ubequad >0 \b%16.16llx +# >>>>>>>>>>>>>>>>>>0x4A8 ubequad >0 \b%16.16llx +# look for end of FAT +#>>0x420 search/512/s \xff\xff FAT END +# Physical block number of FAT header +#>0x40 ubyte x \b, FAT at phy. block %u +>0x40 ubyte x +>>(0x40.b*512) ubyte x +# 1st FAT block +>>>&511 use garmin-fat +# 2nd FAT block +>>>&1023 use garmin-fat +# 3th FAT block +>>>&1535 use garmin-fat +# 4th FAT block +>>>&2047 use garmin-fat +# ... xth FAT block +# +# 314 zeros but not in vpm and also gmaptz.img +>0x84 uquad !0 \b, at 0x84 0x%llx +# display FileAllocationTable block entry in garmin map +0 name garmin-fat +>0 ubyte x \b; +# sub file part; 0x0003 seems to be garbage +>0x10 uleshort !0 next 0x%4.4x +>0x10 uleshort =0 +# fat flag 0~dummy block 1~true sub file +>>0 ubyte !1 flag %u +>>0 ubyte =1 +# sub-file name like MAKEGMAP 12345678 +>>>0x1 string x %.8s +# sub-file typ like RGN TRE MDR LBL +>>>0x9 string x \b.%.3s +# size of sub file +>>>0xC ulelong x \b, %u bytes +# 32-bit block sequence numbers +#>>>0x20 ubequad x \b, seq. 0x%16.16llx + +# display date stored inside Garmin maps like yyyy-mm-dd h:mm:ss +0 name garmin-date +# year like 2018 +>0 uleshort x \b, created %u +# month (0-11) +>2 ubyte x \b-%.2u +# day (1-31) +>3 ubyte x \b-%.2u +# hour (0-23) +>4 ubyte x %u +# minute (0-59) +>5 ubyte x \b:%.2u +# second (0-59) +>6 ubyte x \b:%.2u + +# Summary: Garmin Map subfiles +# From: Joerg Jenderek +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/IMG_File_Format +# Garmin Common Header +2 string GARMIN\ +# skip ASCII text by checking for low header length +>0 uleshort <0x1000 Garmin map, +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/GMP_Subfile_Format +>>9 string GMP subtile +!:mime application/x-garmin-gpm +!:ext gmp +# copyright message +>>>(0.s) string x %s +>>>0x0E use garmin-date +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/MDR_Subfile_Format +# This contains the searchable address table used for finding routing destinations +>>9 string MDR address table +!:mime application/x-garmin-mdr +!:ext mdr +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/NOD_Subfile_Format +# http://svn.parabola.me.uk/display/trunk/doc/nod.txt +# This contains the routing information +>>9 string NOD routing +!:mime application/x-garmin-nod +!:ext nod +>>>0x0E use garmin-date +#>>>0x15 ulelong x \b, at 0x%x +#>>>0x19 ulelong x 0x%x bytes NOD1 +#>>>0x25 ulelong x \b, at 0x%x +#>>>0x29 ulelong x 0x%x bytes NOD2 +#>>>0x31 ulelong x \b, at 0x%x +#>>>0x35 ulelong x 0x%x bytes NOD3 +# URL: http://www.pinns.co.uk/osm/net.html +# routable highways (length, direction, allowed speed,house address information) +>>9 string NET highways +!:mime application/x-garmin-net +!:ext net +#>>>0x15 ulelong x \b, at 0x%x +#>>>0x19 ulelong x 0x%x bytes NET1 +#>>>0x22 ulelong >0 +#>>>>0x1E ulelong x \b, at 0x%x +#>>>>0x22 ulelong x 0x%x bytes NET2 +#>>>0x2B ulelong >0 +#>>>>0x27 ulelong x \b, at 0x%x +#>>>>0x2B ulelong x 0x%x bytes NET3 +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/LBL_Subfile_Format +>>9 string LBL labels +!:mime application/x-garmin-lbl +!:ext lbl +>>>(0.s) string x %s +# Label coding type 6h 9h and ah +>>>0x1E ubyte x \b, coding type 0x%x +#>>>0x15 ulelong x \b, at 0x%x +#>>>0x19 ulelong x 0x%x bytes LBL1 +#>>>0x1F ulelong x \b, at 0x%x +#>>>0x23 ulelong x 0x%x bytes LBL2 +#>>>0x2D ulelong x \b, at 0x%x +#>>>0x31 ulelong x 0x%x bytes LBL3 +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/SRT_Subfile_Format +# A lookup table of the chars in the map's codepage, and their collating sequence +>>9 string SRT sort table +!:mime application/x-garmin-srt +!:ext srt +>>>0x0E use garmin-date +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/TRE_Subfile_Format +>>9 string TRE tree +!:mime application/x-garmin-tre +!:ext tre +# title like City Nav Europe NTU 2019.2 Basemap +# or OSM Street map +>>>(0.s) string x %s +# 2nd title like Copyright 1995-2018 by GARMIN Corporation. +# or http://www.openstreetmap.org/ +>>>>&1 string x %s +>>>0x0E use garmin-date +#>>>0x21 ulelong x \b, at 0x%x +#>>>0x25 ulelong x 0x%x bytes TRE1 +#>>>0x29 ulelong x \b, at 0x%x +#>>>0x2D ulelong x 0x%x bytes TRE2 +#>>>0x31 ulelong x \b, at 0x%x +#>>>0x35 ulelong x 0x%x bytes TRE3 +# Copyright record size +#>>>0x39 uleshort x \b, copyright record size %u +# Map ID +>>>0x74 ulelong x \b, ID 0x%x +# URL: https://www.gpspower.net/garmin-tutorials/353310-basecamp-installing-free-desktop-map.html +# For road traffic information service (RDS/TMS/TMC). Commonly seen in City Navigator maps +>>9 string TRF traffic, +!:mime application/x-garmin-trf +!:ext trf +# city/region like Preitenegg +>>>(0.s+1) string x 1st %s +# highway part like L606/L148 +>>>>&1 string x %s +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/Format +# Reference: http://www.pinns.co.uk/osm/typformat.html +# customize the appearance of objects. For GPS and MapSource/Qlandkarte better looking maps +>>9 string TYP types +!:mime application/x-garmin-typ +!:ext typ +>>>0x0E use garmin-date +# character set 1252 65001~UTF8 +>>>0x15 uleshort x \b, code page %u +# POIs +#>>>0x17 ulelong x \b, at 0x%x +#>>>0x1B ulelong x 0x%x bytes TYP1 +# extra pois +#>>>0x5B ulelong x \b, at 0x%x +#>>>0x5F ulelong x 0x%x bytes TYP8 +# URL: https://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/RGN_Subfile_Format +# http://www.pinns.co.uk/osm/RGN.html +# region data used by the Garmin software +>>9 string RGN region +!:mime application/x-garmin-rgn +!:ext rgn +# POIs,Indexed POIs,Polylines or Polygons or first map level +#>>>0x15 ulelong x \b, at 0x%x +#>>>0x19 ulelong x 0x%x bytes RGN1 +# polygons with extended types +#>>>0x21 ulelong >0 +#>>>>0x1D ulelong x \b, at 0x%x +#>>>>0x21 ulelong x 0x%x bytes RGN2 +# polylines with extended types +#>>>0x3D ulelong >0 +#>>>>0x39 ulelong x \b, at 0x%x +#>>>>0x3D ulelong x 0x%x bytes RGN3 +# extended POIs +#>>>0x59 ulelong >0 +#>>>>0x55 ulelong x \b, at 0x%x +#>>>>0x59 ulelong x 0x%x bytes RGN3 +#>>9 default x unknown map type +# Header length; GMP:31h 35h 3Dh,MDR:11Eh 238h 2C4h 310h,NOD:3Fh 7Fh,NET:64h, +# LBL:2A9h,SRT:1Dh 25h 27h,TRE:CFh 135h,TRF:5Ah,TYP:5Bh 6Eh 7Ch AEh,RGN:7Dh +>>0 uleshort x \b, header length 0x%x + # TOM TOM GPS watches ttbin files: -# http://github.com/ryanbinns/ttwatch/tree/master/ttbin +# https://github.com/ryanbinns/ttwatch/tree/master/ttbin # From: Daniel Lenski 0 byte 0x20 >1 leshort 0x0007 diff --git a/magic/Magdir/marc21 b/magic/Magdir/marc21 index fd509ec15245..bb4998ec0451 100644 --- a/magic/Magdir/marc21 +++ b/magic/Magdir/marc21 @@ -5,7 +5,7 @@ # # MARC21 formats are for the representation and communication # of bibliographic and related information in machine-readable -# form. For more info, see http://www.loc.gov/marc/ +# form. For more info, see https://www.loc.gov/marc/ # leader position 20-21 must be 45 diff --git a/magic/Magdir/matroska b/magic/Magdir/matroska index 0c0e29985f45..271af556aafb 100644 --- a/magic/Magdir/matroska +++ b/magic/Magdir/matroska @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: matroska,v 1.8 2013/02/08 17:25:16 christos Exp $ +# $File: matroska,v 1.9 2019/04/19 00:42:27 christos Exp $ # matroska: file(1) magic for Matroska files # -# See http://www.matroska.org/ +# See https://www.matroska.org/ # # EBML id: diff --git a/magic/Magdir/mercurial b/magic/Magdir/mercurial index 8bc5a5471516..b8f3cddb3601 100644 --- a/magic/Magdir/mercurial +++ b/magic/Magdir/mercurial @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: mercurial,v 1.4 2009/09/19 16:28:10 christos Exp $ +# $File: mercurial,v 1.5 2019/04/19 00:42:27 christos Exp $ # mercurial: file(1) magic for Mercurial changeset bundles -# http://www.selenic.com/mercurial/wiki/ +# https://www.selenic.com/mercurial/wiki/ # # Jesse Glick (jesse.glick@sun.com) # diff --git a/magic/Magdir/metastore b/magic/Magdir/metastore index 53f5b37c936a..e64e70440ba5 100644 --- a/magic/Magdir/metastore +++ b/magic/Magdir/metastore @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: metastore,v 1.2 2017/03/17 21:35:28 christos Exp $ +# $File: metastore,v 1.3 2019/04/19 00:42:27 christos Exp $ # metastore: file(1) magic for metastore files # From: Thomas Wissen -# see http://david.hardeman.nu/software.php#metastore +# see https://david.hardeman.nu/software.php#metastore 0 string MeTaSt00r3 Metastore data file, >10 bequad x version %0llx diff --git a/magic/Magdir/microfocus b/magic/Magdir/microfocus index e5b247cd3d90..93e39aa1bcaa 100644 --- a/magic/Magdir/microfocus +++ b/magic/Magdir/microfocus @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: microfocus,v 1.2 2017/03/17 21:35:28 christos Exp $ +# $File: microfocus,v 1.3 2019/04/19 00:42:27 christos Exp $ # Micro Focus COBOL data files. -# http://documentation.microfocus.com/help/index.jsp?topic=\ +# https://documentation.microfocus.com/help/index.jsp?topic=\ # %2FGUID-0E0191D8-C39A-44D1-BA4C-D67107BAF784%2FHRFLRHFILE05.html # http://www.cobolproducts.com/datafile/data-viewer.html # https://github.com/miracle2k/mfcobol-export diff --git a/magic/Magdir/misctools b/magic/Magdir/misctools index 3ce91b560bf7..cef1da5f4e5f 100644 --- a/magic/Magdir/misctools +++ b/magic/Magdir/misctools @@ -1,6 +1,6 @@ #----------------------------------------------------------------------------- -# $File: misctools,v 1.17 2017/03/17 21:35:28 christos Exp $ +# $File: misctools,v 1.18 2019/04/19 00:42:27 christos Exp $ # misctools: file(1) magic for miscellaneous UNIX tools. # 0 search/1 %%!! X-Post-It-Note text @@ -8,7 +8,7 @@ !:mime text/calendar # updated by Joerg Jenderek at Apr 2015 # Extension: .vcf -# http://en.wikipedia.org/wiki/VCard +# https://en.wikipedia.org/wiki/VCard 0 string/c BEGIN:VCARD vCard visiting card # deprecated #!:mime text/x-vcard @@ -36,7 +36,7 @@ # "Windows Minidump" by TrID # ./misctools (version 5.25) labeled the entry as "MDMP crash report data" 0 string MDMP Mini DuMP crash report -# http://filext.com/file-extension/DMP +# https://filext.com/file-extension/DMP !:mime application/x-dmp !:ext dmp/mdmp # The high-order word is an internal value that is implementation specific. diff --git a/magic/Magdir/modem b/magic/Magdir/modem index d988e903bfcf..c7a53ee436fe 100644 --- a/magic/Magdir/modem +++ b/magic/Magdir/modem @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: modem,v 1.8 2017/03/17 21:35:28 christos Exp $ +# $File: modem,v 1.9 2019/04/19 00:42:27 christos Exp $ # modem: file(1) magic for modem programs # # From: Florian La Roche <florian@knorke.saar.de> @@ -11,7 +11,7 @@ # Summary: CCITT Group 3 Facsimile in "raw" form (i.e. no header). # Modified by: Joerg Jenderek # URL: https://de.wikipedia.org/wiki/Fax -# Reference: http://web.archive.org/web/20020628195336/http://www.netnam.vn/unescocourse/computervision/104.htm +# Reference: https://web.archive.org/web/20020628195336/http://www.netnam.vn/unescocourse/computervision/104.htm # GRR: EOL of G3 is too general as it catches also TrueType fonts, Postscript PrinterFontMetric, others 0 short 0x0100 # 16 0-bits near beginning like True Type fonts *.ttf, Postscript PrinterFontMetric *.pfm, FTYPE.HYPERCARD, XFER diff --git a/magic/Magdir/mozilla b/magic/Magdir/mozilla index 2b1b1dcbfca8..bc6b6a66b91e 100644 --- a/magic/Magdir/mozilla +++ b/magic/Magdir/mozilla @@ -1,13 +1,28 @@ #------------------------------------------------------------------------------ -# $File: mozilla,v 1.8 2018/01/17 12:08:36 christos Exp $ +# $File: mozilla,v 1.10 2019/04/19 00:42:27 christos Exp $ # mozilla: file(1) magic for Mozilla XUL fastload files # (XUL.mfasl and XPC.mfasl) -# URL: http://www.mozilla.org/ +# URL: https://www.mozilla.org/ # From: Josh Triplett <josh@freedesktop.org> 0 string XPCOM\nMozFASL\r\n\x1A Mozilla XUL fastload data +# Probably the next magic line contains misspelled "mozLz40\0" 0 string mozLz4a Mozilla lz4 compressed bookmark data +# From: Joerg Jenderek +# URL: https://lz4.github.io/lz4/ +# Reference: https://github.com/avih/dejsonlz4/archive/master.zip/ +# dejsonlz4-master\src\dejsonlz4.c +# Note: mostly JSON compressed with a non-standard LZ4 header +# can be unpacked by dejsonlz4 but not lz4 programm. +0 string mozLz40\0 Mozilla lz4 compressed data +!:mime application/x-lz4+json +# mozlz4 extension seems to be used for search/store, while jsonlz4 for bookmarks +!:ext jsonlz4/mozlz4 +# decomp_size +>8 ulelong x \b, originally %u bytes +# lz4 data +#>12 ubequad x \b, lz4 data 0x%16.16llx # From: Joerg Jenderek # URL: https://en.wikipedia.org/wiki/Firefox_4 diff --git a/magic/Magdir/msdos b/magic/Magdir/msdos index 9e395b41cfd5..eda0ddbb0d8e 100644 --- a/magic/Magdir/msdos +++ b/magic/Magdir/msdos @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: msdos,v 1.124 2018/07/10 04:05:50 christos Exp $ +# $File: msdos,v 1.128 2019/04/19 00:42:27 christos Exp $ # msdos: file(1) magic for MS-DOS files # @@ -9,12 +9,16 @@ 0 string/t @ >1 string/cW \ echo\ off DOS batch file text !:mime text/x-msdos-batch +!:ext bat >1 string/cW echo\ off DOS batch file text !:mime text/x-msdos-batch +!:ext bat >1 string/cW rem DOS batch file text !:mime text/x-msdos-batch +!:ext bat >1 string/cW set\ DOS batch file text !:mime text/x-msdos-batch +!:ext bat # OS/2 batch files are REXX. the second regex is a bit generic, oh well @@ -49,6 +53,9 @@ # All non-DOS EXE extensions have the relocation table more than 0x40 bytes into the file. >0x18 leshort <0x40 MS-DOS executable !:mime application/x-dosexec +# Windows and later versions of DOS will allow .EXEs to be named with a .COM +# extension, mostly for compatibility's sake. +!:ext exe/com # These traditional tests usually work but not always. When test quality support is # implemented these can be turned on. #>>0x18 leshort 0x1c (Borland compiler) @@ -67,9 +74,33 @@ >>>(0x3c.l+24) default x Unknown PE signature >>>>&0 leshort x 0x%x >>>(0x3c.l+22) leshort&0x2000 >0 (DLL) ->>>(0x3c.l+92) leshort 1 (native) ->>>(0x3c.l+92) leshort 2 (GUI) ->>>(0x3c.l+92) leshort 3 (console) +>>>(0x3c.l+92) leshort 1 +# Native PEs include ntoskrnl.exe, hal.dll, smss.exe, autochk.exe, and all the +# drivers in Windows/System32/drivers/*.sys. +>>>>(0x3c.l+22) leshort&0x2000 >0 (native) +!:ext dll/sys +>>>>(0x3c.l+22) leshort&0x2000 0 (native) +!:ext exe/sys +>>>(0x3c.l+92) leshort 2 +>>>>(0x3c.l+22) leshort&0x2000 >0 (GUI) +# These could probably be at least partially distinguished from one another by +# looking for specific exported functions. +# CPL: Control Panel item +# TLB: Type library +# OCX: OLE/ActiveX control +# ACM: Audio compression manager codec +# AX: DirectShow source filter +# IME: Input method editor +!:ext dll/cpl/tlb/ocx/acm/ax/ime +>>>>(0x3c.l+22) leshort&0x2000 0 (GUI) +# Screen savers typically include code from the scrnsave.lib static library, but +# that's not guaranteed. +!:ext exe/scr +>>>(0x3c.l+92) leshort 3 +>>>>(0x3c.l+22) leshort&0x2000 >0 (console) +!:ext dll/cpl/tlb/ocx/acm/ax/ime +>>>>(0x3c.l+22) leshort&0x2000 0 (console) +!:ext exe/com >>>(0x3c.l+92) leshort 7 (POSIX) >>>(0x3c.l+92) leshort 9 (Windows CE) >>>(0x3c.l+92) leshort 10 (EFI application) @@ -151,8 +182,16 @@ >>>(0x3c.l+0x36) default x >>>>(0x3c.l+0x36) byte x (unknown OS %x) >>>(0x3c.l+0x36) byte 0x81 for MS-DOS, Phar Lap DOS extender ->>>(0x3c.l+0x0c) leshort&0x8003 0x8002 (DLL) ->>>(0x3c.l+0x0c) leshort&0x8003 0x8001 (driver) +>>>(0x3c.l+0x0c) leshort&0x8000 0x8000 (DLL or font) +# DRV: Driver +# 3GR: Grabber device driver +# CPL: Control Panel Item +# VBX: Visual Basic Extension +# FON: Bitmap font +# FOT: Font resource file +!:ext dll/drv/3gr/cpl/vbx/fon/fot +>>>(0x3c.l+0x0c) leshort&0x8000 0 (EXE) +!:ext exe/scr >>>&(&0x24.s-1) string ARJSFX \b, ARJ self-extracting archive >>>(0x3c.l+0x70) search/0x80 WinZip(R)\ Self-Extractor \b, ZIP self-extracting archive (WinZip) @@ -199,6 +238,11 @@ >>>(0x3c.l+0x0a) leshort 2 for MS Windows >>>(0x3c.l+0x0a) leshort 3 for DOS >>>(0x3c.l+0x0a) leshort 4 for MS Windows (VxD) +# VXD: VxD for Windows 95/98/Me +# 386: VxD for Windows 2.10, 3.0, 3.1x +# PDR: Port driver +# MPD: Miniport driver (?) +!:ext vxd/386/pdr/mpd >>>(&0x7c.l+0x26) string UPX \b, UPX compressed >>>&(&0x54.l-3) string UNACE \b, ACE self-extracting archive @@ -207,6 +251,7 @@ >>0x3c lelong >0x20000000 >>>(4.s*512) leshort !0x014c \b, MZ for MS-DOS !:mime application/x-dosexec +!:ext exe/com # header data too small for extended executable >2 long !0 >>0x18 leshort <0x40 @@ -305,8 +350,8 @@ >>49824 leshort =1 \b, 1 file >>49824 leshort >1 \b, %u files -# added by Joerg Jenderek of http://www.freedos.org/software/?prog=kc -# and http://www.freedos.org/software/?prog=kpdos +# added by Joerg Jenderek of https://www.freedos.org/software/?prog=kc +# and https://www.freedos.org/software/?prog=kpdos # for FreeDOS files like KEYBOARD.SYS, KEYBRD2.SYS, KEYBRD3.SYS, *.KBD 0 string/b KCF FreeDOS KEYBoard Layout collection # only version=0x100 found @@ -448,6 +493,8 @@ 0 name msdos-com >0 byte x DOS executable (COM) +!:mime application/x-dosexec +!:ext com >6 string SFX\ of\ LHarc \b, %s >0x1FE leshort 0xAA55 \b, boot code >85 string UPX \b, UPX compressed @@ -484,11 +531,11 @@ # modified by Joerg Jenderek # syslinux COM32 or COM32R executable >>1 lelong&0xFFFFFFFe 0x21CD4CFe COM executable (32-bit COMBOOT -# http://www.syslinux.org/wiki/index.php/Comboot_API +# https://www.syslinux.org/wiki/index.php/Comboot_API # Since version 5.00 c32 modules switched from the COM32 object format to ELF !:mime application/x-c32-comboot-syslinux-exec !:ext c32 -# http://syslinux.zytor.com/comboot.php +# https://syslinux.zytor.com/comboot.php # older syslinux version ( <4 ) # (32-bit COMBOOT) programs *.C32 contain 32-bit code and run in flat-memory 32-bit protected mode # start with assembler instructions mov eax,21cd4cffh @@ -514,41 +561,75 @@ 0 string/b \x81\xfc >4 string \x77\x02\xcd\x20\xb9 >>36 string UPX! FREE-DOS executable (COM), UPX compressed +!:mime application/x-dosexec +!:ext com 252 string Must\ have\ DOS\ version DR-DOS executable (COM) +!:mime application/x-dosexec +!:ext com # added by Joerg Jenderek at Oct 2008 # GRR search is not working #34 search/2 UPX! FREE-DOS executable (COM), UPX compressed 34 string UPX! FREE-DOS executable (COM), UPX compressed +!:mime application/x-dosexec +!:ext com 35 string UPX! FREE-DOS executable (COM), UPX compressed +!:mime application/x-dosexec +!:ext com # GRR search is not working #2 search/28 \xcd\x21 COM executable for MS-DOS #WHICHFAT.cOM 2 string \xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com #DELTREE.cOM DELTREE2.cOM 4 string \xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com #IFMEMDSK.cOM ASSIGN.cOM COMP.cOM 5 string \xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com #DELTMP.COm HASFAT32.cOM 7 string \xcd\x21 >0 byte !0xb8 COM executable for DOS +!:mime application/x-dosexec +!:ext com #COMP.cOM MORE.COm 10 string \xcd\x21 >5 string !\xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com #comecho.com 13 string \xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com #HELP.COm EDIT.coM 18 string \xcd\x21 COM executable for MS-DOS +!:mime application/x-dosexec +!:ext com #NWRPLTRM.COm 23 string \xcd\x21 COM executable for MS-DOS +!:mime application/x-dosexec +!:ext com #LOADFIX.cOm LOADFIX.cOm 30 string \xcd\x21 COM executable for MS-DOS +!:mime application/x-dosexec +!:ext com #syslinux.com 3.11 70 string \xcd\x21 COM executable for DOS +!:mime application/x-dosexec +!:ext com # many compressed/converted COMs start with a copy loop instead of a jump 0x6 search/0xa \xfc\x57\xf3\xa5\xc3 COM executable for MS-DOS +!:mime application/x-dosexec +!:ext com 0x6 search/0xa \xfc\x57\xf3\xa4\xc3 COM executable for DOS +!:mime application/x-dosexec +!:ext com >0x18 search/0x10 \x50\xa4\xff\xd5\x73 \b, aPack compressed 0x3c string W\ Collis\0\0 COM executable for MS-DOS, Compack compressed +!:mime application/x-dosexec +!:ext com # FIXME: missing diet .com compression # miscellaneous formats @@ -843,8 +924,9 @@ # skip remaining worksheets, because valid only for DIB image (40) or PNG image (\x89PNG) >>(18.l) ulelong x MS Windows >>>0 ubelong 0x00000100 icon resource -#!:mime image/vnd.microsoft.icon -!:mime image/x-icon +# https://www.iana.org/assignments/media-types/image/vnd.microsoft.icon +!:mime image/vnd.microsoft.icon +#!:mime image/x-icon !:ext ico >>>>4 uleshort x - %d icon # plural s @@ -890,7 +972,8 @@ #>12 ulelong x \b, offset 0x%x # PNG header (\x89PNG) >(12.l) ubelong =0x89504e47 ->>&-4 indirect x \b with +# 1 space char after "with" to get phrase "with PNG image" by magic in ./images +>>&-4 indirect x \b with # DIB image >(12.l) ubelong !0x89504e47 #>>&-4 use dib-image @@ -959,10 +1042,11 @@ 0 lelong 0x4C >4 lelong 0x00021401 Windows shortcut file -# .PIF files added by Joerg Jenderek from http://smsoft.ru/en/pifdoc.htm +# .PIF files added by Joerg Jenderek from https://smsoft.ru/en/pifdoc.htm # only for windows versions equal or greater 3.0 0x171 string MICROSOFT\ PIFEX\0 Windows Program Information File !:mime application/x-dosexec +!:ext pif #>2 string >\0 \b, Title:%.30s >0x24 string >\0 \b for %.63s >0x65 string >\0 \b, directory=%.64s @@ -1010,7 +1094,7 @@ # Norton Guide (.NG , .HLP) files added by Joerg Jenderek from source NG2HTML.C # of http://www.davep.org/norton-guides/ng2h-105.tgz -# http://en.wikipedia.org/wiki/Norton_Guides +# https://en.wikipedia.org/wiki/Norton_Guides 0 string NG\0\001 # only value 0x100 found at offset 2 >2 ulelong 0x00000100 Norton Guide @@ -1022,7 +1106,7 @@ >>114 string >\0 %-.66s # 4DOS help (.HLP) files added by Joerg Jenderek from source TPHELP.PAS -# of http://www.4dos.info/ +# of https://www.4dos.info/ # pointer,HelpID[8]=4DHnnnmm 0 ulelong 0x48443408 4DOS help file >4 string x \b, version %-4.4s @@ -1098,7 +1182,7 @@ !:mime application/vnd.ms-cab-compressed !:ext cab -# http://support.microsoft.com/kb/934307/en-US +# https://support.microsoft.com/kb/934307/en-US # All inspected MSU contain a file with name WSUSSCAN.cab # that is called "Windows Update meta data" by Microsoft >>>&-1 string/c wsusscan.cab \b, Microsoft Standalone Update @@ -1119,7 +1203,7 @@ # or Windows 8 Desktop Theme Pack like in PanoramicGlaciers.deskthemepack >>>>>&0 string/c theme \b, Windows !:mime application/x-windows-themepack -# http://www.drewkeller.com/content/using-theme-both-windows-7-and-windows-8 +# https://www.drewkeller.com/content/using-theme-both-windows-7-and-windows-8 # 1st member Panoramic.theme or Panoramas.theme implies Windows 8-10 Theme Pack # with MTSM=RJSPBS in [MasterThemeSelector] inside *.theme >>>>>>(16.l+16) string =Panoram 8 @@ -1329,7 +1413,7 @@ >>48 string x version %.3s # Type: Microsoft Document Imaging Format (.mdi) -# URL: http://en.wikipedia.org/wiki/Microsoft_Document_Imaging_Format +# URL: https://en.wikipedia.org/wiki/Microsoft_Document_Imaging_Format # From: Daniele Sempione <scrows@oziosi.org> # Too weak (EP) #0 short 0x5045 Microsoft Document Imaging Format @@ -1343,10 +1427,6 @@ # From: Dr. Jesus <j@hug.gs> 0 string/b B000FF\n Windows Embedded CE binary image -# Windows Imaging (WIM) Image -0 string/b MSWIM\000\000\000 Windows imaging (WIM) image -0 string/b WLPWM\000\000\000 Windows imaging (WIM) image, wimlib pipable format - # The second byte of these signatures is a file version; I don't know what, # if anything, produced files with version numbers 0-2. # From: John Elliott <johne@seasip.demon.co.uk> diff --git a/magic/Magdir/msooxml b/magic/Magdir/msooxml index afbc95aca6e3..194cf53fe249 100644 --- a/magic/Magdir/msooxml +++ b/magic/Magdir/msooxml @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: msooxml,v 1.8 2018/05/24 18:11:17 christos Exp $ +# $File: msooxml,v 1.12 2019/04/19 00:42:27 christos Exp $ # msooxml: file(1) magic for Microsoft Office XML # From: Ralf Brown <ralf.brown@gmail.com> @@ -25,20 +25,21 @@ !:strength +10 # make sure the first file is correct >0x1E use msooxml ->0x1E regex \\[Content_Types\\]\\.xml|_rels/\\.rels +>0x1E regex \\[Content_Types\\]\\.xml|_rels/\\.rels|docProps # skip to the second local file header # since some documents include a 520-byte extra field following the file # header, we need to scan for the next header ->>(18.l+49) search/2000 PK\003\004 +>>(18.l+49) search/6000 PK\003\004 # now skip to the *third* local file header; again, we need to scan due to a # 520-byte extra field following the file header ->>>&26 search/1000 PK\003\004 +>>>&26 search/6000 PK\003\004 # and check the subdirectory name to determine which type of OOXML # file we have. Correct the mimetype with the registered ones: -# http://technet.microsoft.com/en-us/library/cc179224.aspx +# https://technet.microsoft.com/en-us/library/cc179224.aspx >>>>&26 use msooxml >>>>&26 default x # OpenOffice/Libreoffice orders ZIP entry differently, so check the 4th file ->>>>>&26 search/1000 PK\003\004 +>>>>>&26 search/6000 PK\003\004 >>>>>>&26 use msooxml >>>>>>&26 default x Microsoft OOXML +>>>>>&26 default x Microsoft OOXML diff --git a/magic/Magdir/msvc b/magic/Magdir/msvc index 13f847fbb20e..8cf5c166d3f1 100644 --- a/magic/Magdir/msvc +++ b/magic/Magdir/msvc @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: msvc,v 1.9 2017/08/02 08:15:20 christos Exp $ +# $File: msvc,v 1.10 2018/10/01 19:14:03 christos Exp $ # msvc: file(1) magic for msvc # "H. Nanosecond" <aldomel@ix.netcom.com> # Microsoft visual C @@ -55,7 +55,7 @@ >>>0x32 leshort x \b*%d bytes # Reference: https://github.com/Microsoft/vstest/pull/856/commits/fdc7a9f074ca5a8dfeec83b1be9162bf0cf4000d -0 string/c bsjb\001\000\001\000\000\000\000\000\f\000\000\000pdb\ v1.0 Microsoft Rosyln C# debugging symbols version 1.0 +0 string/c bsjb\001\000\001\000\000\000\000\000\f\000\000\000pdb\ v1.0 Microsoft Roslyn C# debugging symbols version 1.0 #.sbr 0 string \000\002\000\007\000 MSVC .sbr diff --git a/magic/Magdir/neko b/magic/Magdir/neko index ac5ff354ec12..6bedc22a5a38 100644 --- a/magic/Magdir/neko +++ b/magic/Magdir/neko @@ -1,9 +1,9 @@ #------------------------------------------------------------ -# $File: neko,v 1.1 2009/11/10 20:36:10 christos Exp $ +# $File: neko,v 1.2 2019/04/19 00:42:27 christos Exp $ # From: Mikhail Gusarov <dottedmag@dottedmag.net> -# NekoVM (http://nekovm.org/) bytecode +# NekoVM (https://nekovm.org/) bytecode 0 string NEKO NekoVM bytecode >4 lelong x (%d global symbols, >8 lelong x %d global fields, diff --git a/magic/Magdir/netbsd b/magic/Magdir/netbsd index 4b1a5dd0f933..77e64f0b2e3f 100644 --- a/magic/Magdir/netbsd +++ b/magic/Magdir/netbsd @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: netbsd,v 1.25 2017/09/28 02:37:47 christos Exp $ +# $File: netbsd,v 1.26 2019/01/01 03:11:23 christos Exp $ # netbsd: file(1) magic for NetBSD objects # # All new-style magic numbers are in network byte order. @@ -9,184 +9,149 @@ # in aout. # -0 belong&0377777777 041400413 a.out NetBSD/i386 demand paged +0 name netbsd-detail +>20 lelong x @%#x +>4 lelong >0 \b+T=%d +>8 lelong >0 \b+D=%d +>12 lelong >0 \b+B=%d +>16 lelong >0 \b+S=%d +>24 lelong >0 \b+TR=%d +>28 lelong >0 \b+TD=%d + +0 name netbsd-4096 >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped -0 belong&0377777777 041400410 a.out NetBSD/i386 pure ->0 byte &0x80 dynamically linked executable + +0 name netbsd-8192 +>0 byte &0x80 +>>20 lelong <8192 shared library +>>20 lelong =8192 dynamically linked executable +>>20 lelong >8192 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped -0 belong&0377777777 041400407 a.out NetBSD/i386 +>0 use netbsd-detail + +0 name netbsd-normal >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped -0 belong&0377777777 041400507 a.out NetBSD/i386 core +>0 use netbsd-detail + +0 name netbsd-pure +>0 byte &0x80 dynamically linked executable +>0 byte ^0x80 executable +>16 lelong >0 not stripped +>0 use netbsd-detail + +0 name netbsd-core >12 string >\0 from '%s' >32 lelong !0 (signal %d) +0 belong&0377777777 041400413 a.out NetBSD/i386 demand paged +>0 use netbsd-4096 + +0 belong&0377777777 041400410 a.out NetBSD/i386 pure +>0 use netbsd-pure + +0 belong&0377777777 041400407 a.out NetBSD/i386 +>0 use netbsd-normal + +0 belong&0377777777 041400507 a.out NetBSD/i386 core +>0 use netbsd-core + 0 belong&0377777777 041600413 a.out NetBSD/m68k demand paged ->0 byte &0x80 ->>20 belong <8192 shared library ->>20 belong =8192 dynamically linked executable ->>20 belong >8192 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-8192 + 0 belong&0377777777 041600410 a.out NetBSD/m68k pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-pure + 0 belong&0377777777 041600407 a.out NetBSD/m68k ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 belong !0 executable ->>20 belong =0 object file ->16 belong >0 not stripped +>0 use \^netbsd-normal + 0 belong&0377777777 041600507 a.out NetBSD/m68k core ->12 string >\0 from '%s' ->32 belong !0 (signal %d) +>0 use \^netbsd-core 0 belong&0377777777 042000413 a.out NetBSD/m68k4k demand paged ->0 byte &0x80 ->>20 belong <4096 shared library ->>20 belong =4096 dynamically linked executable ->>20 belong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-4096 + 0 belong&0377777777 042000410 a.out NetBSD/m68k4k pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-pure + 0 belong&0377777777 042000407 a.out NetBSD/m68k4k ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 belong !0 executable ->>20 belong =0 object file ->16 belong >0 not stripped +>0 use \^netbsd-normal + 0 belong&0377777777 042000507 a.out NetBSD/m68k4k core ->12 string >\0 from '%s' ->32 belong !0 (signal %d) +>0 use \^netbsd-core 0 belong&0377777777 042200413 a.out NetBSD/ns32532 demand paged ->0 byte &0x80 ->>20 lelong <4096 shared library ->>20 lelong =4096 dynamically linked executable ->>20 lelong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-4096 + 0 belong&0377777777 042200410 a.out NetBSD/ns32532 pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-pure + 0 belong&0377777777 042200407 a.out NetBSD/ns32532 ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 lelong !0 executable ->>20 lelong =0 object file ->16 lelong >0 not stripped +>0 use netbsd-normal + 0 belong&0377777777 042200507 a.out NetBSD/ns32532 core ->12 string >\0 from '%s' ->32 lelong !0 (signal %d) +>0 use netbsd-core 0 belong&0377777777 045200507 a.out NetBSD/powerpc core ->12 string >\0 from '%s' +>0 use netbsd-core 0 belong&0377777777 042400413 a.out NetBSD/SPARC demand paged ->0 byte &0x80 ->>20 belong <8192 shared library ->>20 belong =8192 dynamically linked executable ->>20 belong >8192 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-8192 + 0 belong&0377777777 042400410 a.out NetBSD/SPARC pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use \^netbsd-pure + 0 belong&0377777777 042400407 a.out NetBSD/SPARC ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 belong !0 executable ->>20 belong =0 object file ->16 belong >0 not stripped +>0 use \^netbsd-normal + 0 belong&0377777777 042400507 a.out NetBSD/SPARC core ->12 string >\0 from '%s' ->32 belong !0 (signal %d) +>0 use \^netbsd-core 0 belong&0377777777 042600413 a.out NetBSD/pmax demand paged ->0 byte &0x80 ->>20 lelong <4096 shared library ->>20 lelong =4096 dynamically linked executable ->>20 lelong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-4096 + 0 belong&0377777777 042600410 a.out NetBSD/pmax pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use \^netbsd-pure + 0 belong&0377777777 042600407 a.out NetBSD/pmax ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 lelong !0 executable ->>20 lelong =0 object file ->16 lelong >0 not stripped +>0 use netbsd-normal + 0 belong&0377777777 042600507 a.out NetBSD/pmax core ->12 string >\0 from '%s' ->32 lelong !0 (signal %d) +>0 use netbsd-core 0 belong&0377777777 043000413 a.out NetBSD/vax 1k demand paged ->0 byte &0x80 ->>20 lelong <4096 shared library ->>20 lelong =4096 dynamically linked executable ->>20 lelong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-4096 + 0 belong&0377777777 043000410 a.out NetBSD/vax 1k pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-pure + 0 belong&0377777777 043000407 a.out NetBSD/vax 1k ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 lelong !0 executable ->>20 lelong =0 object file ->16 lelong >0 not stripped +>0 use netbsd-normal + 0 belong&0377777777 043000507 a.out NetBSD/vax 1k core ->12 string >\0 from '%s' ->32 lelong !0 (signal %d) +>0 use netbsd-core 0 belong&0377777777 045400413 a.out NetBSD/vax 4k demand paged ->0 byte &0x80 ->>20 lelong <4096 shared library ->>20 lelong =4096 dynamically linked executable ->>20 lelong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-4096 + 0 belong&0377777777 045400410 a.out NetBSD/vax 4k pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-pure + 0 belong&0377777777 045400407 a.out NetBSD/vax 4k ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 lelong !0 executable ->>20 lelong =0 object file ->16 lelong >0 not stripped +>0 use netbsd-normal + 0 belong&0377777777 045400507 a.out NetBSD/vax 4k core ->12 string >\0 from '%s' ->32 lelong !0 (signal %d) +>0 use netbsd-core # NetBSD/alpha does not support (and has never supported) a.out objects, # so no rules are provided for them. NetBSD/alpha ELF objects are @@ -199,50 +164,31 @@ >32 lelong !0 (signal %d) 0 belong&0377777777 043400413 a.out NetBSD/mips demand paged ->0 byte &0x80 ->>20 belong <8192 shared library ->>20 belong =8192 dynamically linked executable ->>20 belong >8192 dynamically linked executable ->0 byte ^0x80 executable +>0 use \^netbsd-8192 + >16 belong >0 not stripped 0 belong&0377777777 043400410 a.out NetBSD/mips pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 belong >0 not stripped +>0 use netbsd-pure + 0 belong&0377777777 043400407 a.out NetBSD/mips ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 belong !0 executable ->>20 belong =0 object file ->16 belong >0 not stripped +>0 use netbsd-normal + 0 belong&0377777777 043400507 a.out NetBSD/mips core ->12 string >\0 from '%s' ->32 belong !0 (signal %d) +>0 use netbsd-core 0 belong&0377777777 043600413 a.out NetBSD/arm32 demand paged ->0 byte &0x80 ->>20 lelong <4096 shared library ->>20 lelong =4096 dynamically linked executable ->>20 lelong >4096 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-4096 + 0 belong&0377777777 043600410 a.out NetBSD/arm32 pure ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 executable ->16 lelong >0 not stripped +>0 use netbsd-pure + 0 belong&0377777777 043600407 a.out NetBSD/arm32 ->0 byte &0x80 dynamically linked executable ->0 byte ^0x80 ->>0 byte &0x40 position independent ->>20 lelong !0 executable ->>20 lelong =0 object file ->16 lelong >0 not stripped +>0 use netbsd-normal + # NetBSD/arm26 has always used ELF objects, but it shares a core file # format with NetBSD/arm32. 0 belong&0377777777 043600507 a.out NetBSD/arm core ->12 string >\0 from '%s' ->32 lelong !0 (signal %d) +>0 use netbsd-core # Kernel core dump format 0 belong&0x0000ffff 0x00008fca NetBSD kernel core file diff --git a/magic/Magdir/nitpicker b/magic/Magdir/nitpicker index 48c3d63a288a..bea96c3e745a 100644 --- a/magic/Magdir/nitpicker +++ b/magic/Magdir/nitpicker @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: nitpicker,v 1.7 2017/03/17 21:35:28 christos Exp $ +# $File: nitpicker,v 1.8 2019/04/19 00:42:27 christos Exp $ # nitpicker: file(1) magic for Flowfiles. -# From: Christian Jachmann <C.Jachmann@gmx.net> http://www.nitpicker.de +# From: Christian Jachmann <C.Jachmann@gmx.net> https://www.nitpicker.de 0 string NPFF NItpicker Flow File >4 byte x V%d. >5 byte x %d diff --git a/magic/Magdir/numpy b/magic/Magdir/numpy new file mode 100644 index 000000000000..c1520dd5df93 --- /dev/null +++ b/magic/Magdir/numpy @@ -0,0 +1,9 @@ + +#------------------------------------------------------------------------------ +# $File: numpy,v 1.1 2019/05/09 16:24:36 christos Exp $ +# numpy: file(1) magic for NumPy array binary serialization format +# Reference: https://docs.scipy.org/doc/numpy/reference/generated/numpy.lib.format.html +0 string \x93NUMPY NumPy array, +>6 ubyte x version %d +>7 ubyte x \b.%d, +>8 uleshort x header length %d diff --git a/magic/Magdir/ole2compounddocs b/magic/Magdir/ole2compounddocs index 7b93e613982a..f4046cc97fc0 100644 --- a/magic/Magdir/ole2compounddocs +++ b/magic/Magdir/ole2compounddocs @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: ole2compounddocs,v 1.5 2017/10/27 21:43:23 christos Exp $ +# $File: ole2compounddocs,v 1.6 2019/04/19 00:42:27 christos Exp $ # Microsoft OLE 2 Compound Documents : file(1) magic for Microsoft Structured # storage (https://en.wikipedia.org/wiki/Compound_File_Binary_Format) # Additional tests for OLE 2 Compound Documents should be under this recipe. @@ -15,7 +15,7 @@ # Note: moved & merged Microsoft Office parts from ./msdos Oct 2017 # Update: Joerg Jenderek -# from http://filext.com by Derek M Jones <derek@knosof.co.uk> +# from https://filext.com by Derek M Jones <derek@knosof.co.uk> # False positive with PPT (also currently this string is too long) #0 string/b \xD0\xCF\x11\xE0\xA1\xB1\x1A\xE1\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x3E\x00\x03\x00\xFE\xFF\x09\x00\x06 Microsoft Installer #0 string/b \320\317\021\340\241\261\032\341 Microsoft Office Document diff --git a/magic/Magdir/palm b/magic/Magdir/palm index e852cc718107..8cec9df20db1 100644 --- a/magic/Magdir/palm +++ b/magic/Magdir/palm @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: palm,v 1.13 2014/03/30 21:40:08 christos Exp $ +# $File: palm,v 1.14 2019/04/19 00:42:27 christos Exp $ # palm: file(1) magic for PalmOS {.prc,.pdb}: applications, docfiles, and hacks # # Brian Lalor <blalor@hcirisc.cs.binghamton.edu> @@ -143,8 +143,8 @@ # Palm OS .prc file types 60 string libr # flags, only bit 0 or bit 6 -# http://en.wikipedia.org/wiki/PRC_%28Palm_OS%29 -# http://web.mit.edu/tytso/www/pilot/prc-format.html +# https://en.wikipedia.org/wiki/PRC_%28Palm_OS%29 +# https://web.mit.edu/tytso/www/pilot/prc-format.html >0x20 beshort&0xffbe 0 >>0 string >\0 Palm OS dynamic library data "%s" 60 string ptch Palm OS operating system patch data diff --git a/magic/Magdir/parrot b/magic/Magdir/parrot index 24e9236da429..b2a56c817a23 100644 --- a/magic/Magdir/parrot +++ b/magic/Magdir/parrot @@ -1,7 +1,7 @@ #------------------------------------------------------------------------------ -# $File: parrot,v 1.1 2010/07/08 20:18:40 christos Exp $ +# $File: parrot,v 1.2 2019/04/19 00:42:27 christos Exp $ # parrot: file(1) magic for Parrot Virtual Machine -# URL: http://www.lua.org/ +# URL: https://www.lua.org/ # From: Lubomir Rintel <lkundrak@v3.sk> # Compiled Parrot byte code diff --git a/magic/Magdir/pbf b/magic/Magdir/pbf index 8de6db025047..0ab7a8810103 100644 --- a/magic/Magdir/pbf +++ b/magic/Magdir/pbf @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: pbf,v 1.2 2017/01/18 16:16:21 christos Exp $ +# $File: pbf,v 1.3 2019/04/19 00:42:27 christos Exp $ # file(1) magic(5) data for OpenStreetMap # OpenStreetMap Protocolbuffer Binary Format (.osm.pbf) -# http://wiki.openstreetmap.org/wiki/PBF_Format +# https://wiki.openstreetmap.org/wiki/PBF_Format # From: Markus Heidelberg <markus.heidelberg@web.de> 0 belong&0xfffffff0 0 >4 beshort 0x0A09 diff --git a/magic/Magdir/pc98 b/magic/Magdir/pc98 index 3b995ed013d4..e8f6b8a57aad 100644 --- a/magic/Magdir/pc98 +++ b/magic/Magdir/pc98 @@ -21,7 +21,7 @@ # Maki-chan v2 Graphic format # http://www.jisyo.com/viewer/faq/mag_tech.htm -# http://mooncore.eu/bunny/txt/makichan.htm +# https://mooncore.eu/bunny/txt/makichan.htm # http://metanest.jp/mag/mag.xhtml 0 string/b MAKI02\ \ Maki-chan v2 image, >8 byte x system ID: %c diff --git a/magic/Magdir/pgp b/magic/Magdir/pgp index ecdcf838cc65..acb5395dea57 100644 --- a/magic/Magdir/pgp +++ b/magic/Magdir/pgp @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: pgp,v 1.15 2018/02/24 16:11:23 christos Exp $ +# $File: pgp,v 1.17 2019/04/19 00:42:27 christos Exp $ # pgp: file(1) magic for Pretty Good Privacy -# see http://lists.gnupg.org/pipermail/gnupg-devel/1999-September/016052.html +# see https://lists.gnupg.org/pipermail/gnupg-devel/1999-September/016052.html # # Update: Joerg Jenderek # Note: verified by `gpg -v --debug 0x02 --list-packets < PUBRING263_10.PGP` @@ -74,7 +74,7 @@ # Decode the type of the packet based on it's base64 encoding. # Idea from Mark Martinec # The specification is in RFC 4880, section 4.2 and 4.3: -# http://tools.ietf.org/html/rfc4880#section-4.2 +# https://tools.ietf.org/html/rfc4880#section-4.2 0 name pgp >0 byte 0x67 Reserved (old) @@ -517,9 +517,9 @@ # PGP RSA (e=65537) secret (sub-)key header -0 byte 0x95 PGP Secret Key - +0 byte 0x95 PGP Secret Key - >1 use pgpkey -0 byte 0x97 PGP Secret Sub-key - +0 byte 0x97 PGP Secret Sub-key - >1 use pgpkey 0 byte 0x9d # Update: Joerg Jenderek diff --git a/magic/Magdir/polyml b/magic/Magdir/polyml index 0af9baf8839d..1cc01093e47d 100644 --- a/magic/Magdir/polyml +++ b/magic/Magdir/polyml @@ -1,13 +1,13 @@ #------------------------------------------------------------------------------ -# $File: polyml,v 1.1 2016/02/26 15:52:45 christos Exp $ +# $File: polyml,v 1.2 2019/04/19 00:42:27 christos Exp $ # polyml: file(1) magic for PolyML # # PolyML # MPEG, FLI, DL originally from vax@ccwf.cc.utexas.edu (VaX#n8) # FLC, SGI, Apple originally from Daniel Quinlan (quinlan@yggdrasil.com) -# [0]: http://www.polyml.org/ +# [0]: https://www.polyml.org/ # [1]: https://github.com/polyml/polyml/blob/master/\ # libpolyml/savestate.cpp#L146-L147 # [2]: https://github.com/polyml/polyml/blob/master/\ diff --git a/magic/Magdir/printer b/magic/Magdir/printer index 98fc1dfaad42..e8fccd279717 100644 --- a/magic/Magdir/printer +++ b/magic/Magdir/printer @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: printer,v 1.28 2017/03/17 22:20:22 christos Exp $ +# $File: printer,v 1.29 2019/04/19 00:42:27 christos Exp $ # printer: file(1) magic for printer-formatted files # @@ -40,7 +40,7 @@ # Summary: Adobe's PostScript Printer Description File # Extension: .ppd -# Reference: http://partners.adobe.com/public/developer/en/ps/5003.PPD_Spec_v4.3.pdf, Section 3.8 +# Reference: https://partners.adobe.com/public/developer/en/ps/5003.PPD_Spec_v4.3.pdf, Section 3.8 # Submitted by: Yves Arrouye <arrouye@marin.fdn.fr> # 0 string *PPD-Adobe:\x20 PPD file diff --git a/magic/Magdir/psl b/magic/Magdir/psl index 1f88df106e5a..0296540ce57a 100644 --- a/magic/Magdir/psl +++ b/magic/Magdir/psl @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: psl,v 1.2 2016/07/14 17:34:27 christos Exp $ +# $File: psl,v 1.3 2019/04/19 00:42:27 christos Exp $ # psl: file(1) magic for Public Suffix List representations # From: Daniel Kahn Gillmor <dkg@fifthhorseman.net> # URL: https://publicsuffix.org -# see also: http://thread.gmane.org/gmane.network.dns.libpsl.bugs/162/focus=166 +# see also: https://thread.gmane.org/gmane.network.dns.libpsl.bugs/162/focus=166 0 search/512 \n\n//\ ===BEGIN\ ICANN\ DOMAINS===\n\n Public Suffix List data diff --git a/magic/Magdir/pwsafe b/magic/Magdir/pwsafe index 93dcfb123469..549093f14300 100644 --- a/magic/Magdir/pwsafe +++ b/magic/Magdir/pwsafe @@ -1,14 +1,14 @@ #------------------------------------------------------------------------------ -# $File: pwsafe,v 1.1 2012/10/25 00:12:19 christos Exp $ +# $File: pwsafe,v 1.2 2019/04/19 00:42:27 christos Exp $ # pwsafe: file(1) magic for passwordsafe file # # Password Safe # http://passwordsafe.sourceforge.net/ # file format specs -# http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/formatV3.txt -# V2 http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/formatV2.txt -# V1 http://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/notes.txt +# https://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/formatV3.txt +# V2 https://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/formatV2.txt +# V1 https://passwordsafe.svn.sourceforge.net/viewvc/passwordsafe/trunk/pwsafe/pwsafe/docs/notes.txt # V2 and V1 have no easy identifier that I can find # .psafe3 0 string PWS3 Password Safe V3 database diff --git a/magic/Magdir/python b/magic/Magdir/python index f21ff659b453..acf05dddbaaf 100644 --- a/magic/Magdir/python +++ b/magic/Magdir/python @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: python,v 1.34 2017/08/14 07:40:38 christos Exp $ +# $File: python,v 1.36 2019/04/09 18:28:25 christos Exp $ # python: file(1) magic for python # # Outlook puts """ too for urgent messages @@ -27,7 +27,7 @@ 0 belong 0x160d0d0a python 3.5.1- byte-compiled 0 belong 0x170d0d0a python 3.5.2+ byte-compiled 0 belong 0x330d0d0a python 3.6 byte-compiled -0 belong 0x3e0d0d0a python 3.7 byte-compiled +0 belong 0x420d0d0a python 3.7 byte-compiled 0 search/1/w #!\ /usr/bin/python Python script text executable @@ -45,7 +45,8 @@ # from module.submodule import func1, func2 -0 regex \^from[\040\t\f\r\n]+([A-Za-z0-9_]|\\.)+[\040\t\f\r\n]+import.*$ Python script text executable +0 search/8192 import +>0 regex \^from[\040\t\f\r\n]+([A-Za-z0-9_]|\\.)+[\040\t\f\r\n]+import.*$ Python script text executable !:strength + 15 !:mime text/x-python @@ -63,7 +64,8 @@ !:mime text/x-python # import module [as abrev] -0 regex \^import\ [_[:alpha:]]+\ as\ [[:alpha:]][[:space:]]*$ Python script text executable +0 search/8192 import +>0 regex \^import\ [_[:alpha:]]+\ as\ [[:alpha:]][[:space:]]*$ Python script text executable !:mime text/x-python # comments @@ -86,12 +88,14 @@ !:mime text/x-python # class name[(base classes,)]: [pass] -0 regex \^class\ [_[:alpha:]]+(\\(.*\\))?(\ )*:([\ \t]+pass)?$ Python script text executable +0 search/8192 class +>0 regex \^class\ [_[:alpha:]]+(\\(.*\\))?(\ )*:([\ \t]+pass)?$ Python script text executable !:strength + 15 !:mime text/x-python # def name(*args, **kwargs): -0 regex \^[[:space:]]{0,50}def\ {1,50}[_a-zA-Z]{1,100} ->&0 regex \\(([[:alpha:]*_,\ ]){0,255}\\):$ Python script text executable +0 search/8192 def\ +>0 regex \^[[:space:]]{0,50}def\ {1,50}[_a-zA-Z]{1,100} +>>&0 regex \\(([[:alpha:]*_,\ ]){0,255}\\):$ Python script text executable !:strength + 15 !:mime text/x-python diff --git a/magic/Magdir/qt b/magic/Magdir/qt index 72e6ff3a0d67..83aa124cfd3d 100644 --- a/magic/Magdir/qt +++ b/magic/Magdir/qt @@ -1,9 +1,9 @@ #------------------------------------------------------------------------------ -# $File: qt,v 1.2 2014/12/16 19:49:29 christos Exp $ +# $File: qt,v 1.3 2019/04/19 00:42:27 christos Exp $ # qt: file(1) magic for Qt -# http://doc.qt.io/qt-5/resources.html +# https://doc.qt.io/qt-5/resources.html 0 string \<!DOCTYPE\040RCC\> Qt Resource Collection file # https://qt.gitorious.org/qt/qtbase/source/\ diff --git a/magic/Magdir/revision b/magic/Magdir/revision index 4ae4f75f9bf3..824220a3d295 100644 --- a/magic/Magdir/revision +++ b/magic/Magdir/revision @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: revision,v 1.10 2017/10/19 16:40:37 christos Exp $ +# $File: revision,v 1.11 2019/04/19 00:42:27 christos Exp $ # file(1) magic for revision control files # From Hendrik Scholz <hendrik@scholz.net> 0 string/t /1\ :pserver: cvs password text file @@ -60,7 +60,7 @@ >28 string >\0 (version: %s) # Type: Bazaar revision bundles and merge requests -# URL: http://www.bazaar-vcs.org/ +# URL: https://www.bazaar-vcs.org/ # From: Jelmer Vernooij <jelmer@samba.org> 0 string #\ Bazaar\ revision\ bundle\ v Bazaar Bundle 0 string #\ Bazaar\ merge\ directive\ format Bazaar merge directive diff --git a/magic/Magdir/riff b/magic/Magdir/riff index e11c5536b684..33d439aca7db 100644 --- a/magic/Magdir/riff +++ b/magic/Magdir/riff @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: riff,v 1.33 2017/10/06 01:11:24 christos Exp $ +# $File: riff,v 1.34 2019/04/19 00:42:27 christos Exp $ # riff: file(1) magic for RIFF format # See # -# http://www.seanet.com/users/matts/riffmci/riffmci.htm +# https://www.seanet.com/users/matts/riffmci/riffmci.htm # http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Docs/riffmci.pdf # @@ -73,7 +73,7 @@ # RIFF Palette format # Update: Joerg Jenderek # URL: https://en.wikipedia.org/wiki/Resource_Interchange_File_Format -# Reference: http://worms2d.info/Palette_file +# Reference: https://worms2d.info/Palette_file >8 string PAL\ \b, palette !:mime application/x-riff # color palette by Microsoft Corporation @@ -250,7 +250,7 @@ # MPEG-1 wrapped in a RIFF, apparently >8 string CDXA \b, wrapped MPEG-1 (CDXA) >8 string 4XMV \b, 4X Movie file -# AMV-type AVI file: http://wiki.multimedia.cx/index.php?title=AMV +# AMV-type AVI file: https://wiki.multimedia.cx/index.php?title=AMV >8 string AMV\040 \b, AMV >8 string WEBP \b, Web/P image !:mime image/webp @@ -322,7 +322,7 @@ #------------------------------------------------------------------------------ # MBWF/RF64 -# see EBU TECH 3306 http://tech.ebu.ch/docs/tech/tech3306-2009.pdf +# see EBU TECH 3306 https://tech.ebu.ch/docs/tech/tech3306-2009.pdf 0 string RF64\xff\xff\xff\xffWAVEds64 MBWF/RF64 audio !:mime audio/x-wav >40 search/256 fmt\x20 \b diff --git a/magic/Magdir/rpmsg b/magic/Magdir/rpmsg new file mode 100644 index 000000000000..cbbbb2bc4f52 --- /dev/null +++ b/magic/Magdir/rpmsg @@ -0,0 +1,7 @@ + +#------------------------------------------------------------------------------ +# $File: rpmsg,v 1.1 2019/04/19 00:40:47 christos Exp $ +# rpmsg: file(1) magic for restricted-permission messages (or "rights-protected" messages) +# see https://en.wikipedia.org/wiki/Rpmsg + +0 string \x76\xe8\x04\x60\xc4\x11\xe3\x86 rpmsg Restricted Permission Message diff --git a/magic/Magdir/ruby b/magic/Magdir/ruby index 45a253cb5f69..87af47d933fa 100644 --- a/magic/Magdir/ruby +++ b/magic/Magdir/ruby @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: ruby,v 1.7 2017/08/14 13:39:18 christos Exp $ +# $File: ruby,v 1.9 2019/04/19 00:42:27 christos Exp $ # ruby: file(1) magic for Ruby scripting language -# URL: http://www.ruby-lang.org/ +# URL: https://www.ruby-lang.org/ # From: Reuben Thomas <rrt@sc3d.org> # Ruby scripts @@ -22,9 +22,10 @@ # What looks like ruby, but does not have a shebang # (modules and such) # From: Lubomir Rintel <lkundrak@v3.sk> -0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+' ->0 regex def\ [a-z]|\ do$ ->>&0 regex \^[[:space:]]*end([[:space:]]+[;#].*)?$ Ruby script text +0 search/8192 require +>0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+' +>>0 regex def\ [a-z]|\ do$ +>>>&0 regex \^[[:space:]]*end([[:space:]]+[;#].*)?$ Ruby script text !:strength + 30 !:mime text/x-ruby 0 regex \^[[:space:]]*(class|module)[[:space:]][A-Z] @@ -40,12 +41,15 @@ # Looks for function definition to balance python magic # def name (args) # end -0 regex \^[[:space:]]*def\ [a-z]|def\ [[:alpha:]]+::[a-z] ->&0 regex \^[[:space:]]*end([[:space:]]+[;#].*)?$ Ruby script text +0 search/8192 def\ +>0 regex \^[[:space:]]*def\ [a-z]|def\ [[:alpha:]]+::[a-z] +>>&0 regex \^[[:space:]]*end([[:space:]]+[;#].*)?$ Ruby script text !:strength + 10 !:mime text/x-ruby -0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+' Ruby script text +0 search/8192 require +>0 regex \^[[:space:]]*require[[:space:]]'[A-Za-z_/]+' Ruby script text !:mime text/x-ruby -0 regex \^[[:space:]]*include\ ([A-Z]+[a-z]*(::))+ Ruby script text +0 search/8192 include +>0 regex \^[[:space:]]*include\ ([A-Z]+[a-z]*(::))+ Ruby script text !:mime text/x-ruby diff --git a/magic/Magdir/scientific b/magic/Magdir/scientific index 2e0cf49bee74..0e78712fcab3 100644 --- a/magic/Magdir/scientific +++ b/magic/Magdir/scientific @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: scientific,v 1.12 2017/03/17 22:20:22 christos Exp $ +# $File: scientific,v 1.13 2019/04/19 00:42:27 christos Exp $ # scientific: file(1) magic for scientific formats # # From: Joe Krahn <krahn@niehs.nih.gov> @@ -75,8 +75,8 @@ # PDB: Protein Data Bank files # Adam Buchbinder <adam.buchbinder@gmail.com> # -# http://www.wwpdb.org/documentation/format32/sect2.html -# http://www.ch.ic.ac.uk/chemime/ +# https://www.wwpdb.org/documentation/format32/sect2.html +# https://www.ch.ic.ac.uk/chemime/ # # The PDB file format is fixed-field, 80 columns. From the spec: # diff --git a/magic/Magdir/selinux b/magic/Magdir/selinux index 5f2294654349..89d5f5362933 100644 --- a/magic/Magdir/selinux +++ b/magic/Magdir/selinux @@ -1,6 +1,6 @@ # Type: SE Linux policy modules *.pp reference policy # for Fedora 5 to 9, RHEL5, and Debian Etch and Lenny. -# URL: http://doc.coker.com.au/computers/selinux-magic +# URL: https://doc.coker.com.au/computers/selinux-magic # From: Russell Coker <russell@coker.com.au> 0 lelong 0xf97cff8f SE Linux modular policy diff --git a/magic/Magdir/sendmail b/magic/Magdir/sendmail index 840859c56232..54028fdfe227 100644 --- a/magic/Magdir/sendmail +++ b/magic/Magdir/sendmail @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sendmail,v 1.10 2017/08/13 00:21:47 christos Exp $ +# $File: sendmail,v 1.11 2019/04/19 00:42:27 christos Exp $ # sendmail: file(1) magic for sendmail config files # # XXX - byte order? @@ -12,7 +12,7 @@ # Email_23_f217153422.ts Sendmail frozen configuration # - version \330jK\354 0 byte 046 -# http://www.sendmail.com/sm/open_source/docs/older_release_notes/ +# https://www.sendmail.com/sm/open_source/docs/older_release_notes/ # freezed configuration file (dbm format?) created from sendmal.cf with -bz # by older sendmail. til version 8.6 support for frozen configuration files is removed # valid version numbers look like "7.14.4" and should be similar to output of commands diff --git a/magic/Magdir/sequent b/magic/Magdir/sequent index 780d77e5d6ab..da38de65afc3 100644 --- a/magic/Magdir/sequent +++ b/magic/Magdir/sequent @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sequent,v 1.13 2017/03/17 21:35:28 christos Exp $ +# $File: sequent,v 1.14 2019/04/19 00:42:27 christos Exp $ # sequent: file(1) magic for Sequent machines # # Sequent information updated by Don Dwiggins <atsun!dwiggins>. @@ -30,7 +30,7 @@ 0 leshort 0x32eb SYMMETRY i386 executable (invalid @ 0) >16 lelong >0 not stripped >124 lelong >0 version %d -# http://en.wikipedia.org/wiki/Sequent_Computer_Systems +# https://en.wikipedia.org/wiki/Sequent_Computer_Systems # below test line conflicts with MS-DOS 2.11 floppies and Acronis loader #0 leshort 0x42eb SYMMETRY i386 standalone executable 0 leshort 0x42eb diff --git a/magic/Magdir/sgml b/magic/Magdir/sgml index 4aa6aea281b9..987fe10e08d6 100644 --- a/magic/Magdir/sgml +++ b/magic/Magdir/sgml @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sgml,v 1.38 2017/10/11 11:40:43 christos Exp $ +# $File: sgml,v 1.39 2019/04/19 00:42:27 christos Exp $ # Type: SVG Vectorial Graphics # From: Noel Torres <tecnico@ejerciciosresueltos.com> 0 string \<?xml\ version= @@ -19,7 +19,7 @@ !:mime application/xml-sitemap # OpenStreetMap XML (.osm) -# http://wiki.openstreetmap.org/wiki/OSM_XML +# https://wiki.openstreetmap.org/wiki/OSM_XML # From: Markus Heidelberg <markus.heidelberg@web.de> 0 string \<?xml\ version= >14 regex ['"\ \t]*[0-9.]+['"\ \t]* diff --git a/magic/Magdir/sniffer b/magic/Magdir/sniffer index 34eb7c481ad8..14a23ab5d69d 100644 --- a/magic/Magdir/sniffer +++ b/magic/Magdir/sniffer @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sniffer,v 1.19 2013/01/06 01:11:04 christos Exp $ +# $File: sniffer,v 1.25 2019/05/05 17:03:41 christos Exp $ # sniffer: file(1) magic for packet capture files # # From: guy@alum.mit.edu (Guy Harris) @@ -42,7 +42,7 @@ # Sorry, make that "Network Associates Sniffer capture files." # Sorry, make that "Network General old DOS Sniffer capture files." # -0 string TRSNIFF\ data\ \ \ \ \032 Sniffer capture file +0 string TRSNIFF\040data\040\040\040\040\032 Sniffer capture file >33 byte 2 (compressed) >23 leshort x - version %d >25 leshort x \b.%d @@ -77,6 +77,7 @@ # # "libpcap" capture files. +# https://www.tcpdump.org/manpages/pcap-savefile.5.html # (We call them "tcpdump capture file(s)" for now, as "tcpdump" is # the main program that uses that format, but there are other programs # that use "libpcap", or that use the same capture file format.) @@ -84,6 +85,8 @@ 0 name pcap-be >4 beshort x - version %d >6 beshort x \b.%d +# clear that continuation level match +>20 clear x >20 belong 0 (No link-layer encapsulation >20 belong 1 (Ethernet >20 belong 2 (3Mb Ethernet @@ -114,7 +117,7 @@ >20 belong 108 (OpenBSD loopback >20 belong 109 (OpenBSD IPsec encrypted >20 belong 112 (Cisco HDLC ->20 belong 113 (Linux "cooked" +>20 belong 113 (Linux cooked v1 >20 belong 114 (LocalTalk >20 belong 117 (OpenBSD PFLOG >20 belong 119 (802.11 with Prism header @@ -122,13 +125,21 @@ >20 belong 123 (SunATM >20 belong 127 (802.11 with radiotap header >20 belong 129 (Linux ARCNET +>20 belong 130 (Juniper Multi-Link PPP +>20 belong 131 (Juniper Multi-Link Frame Relay +>20 belong 132 (Juniper Encryption Services PIC +>20 belong 133 (Juniper GGSN PIC +>20 belong 134 (Juniper FRF.16 Frame Relay +>20 belong 135 (Juniper ATM2 PIC +>20 belong 136 (Juniper Advanced Services PIC +>20 belong 137 (Juniper ATM1 PIC >20 belong 138 (Apple IP over IEEE 1394 ->20 belong 139 (MTP2 with pseudo-header ->20 belong 140 (MTP2 ->20 belong 141 (MTP3 ->20 belong 142 (SCCP +>20 belong 139 (SS7 MTP2 with pseudo-header +>20 belong 140 (SS7 MTP2 +>20 belong 141 (SS7 MTP3 +>20 belong 142 (SS7 SCCP >20 belong 143 (DOCSIS ->20 belong 144 (IrDA +>20 belong 144 (Linux IrDA >20 belong 147 (Private use 0 >20 belong 148 (Private use 1 >20 belong 149 (Private use 2 @@ -146,16 +157,41 @@ >20 belong 161 (Private use 14 >20 belong 162 (Private use 15 >20 belong 163 (802.11 with AVS header +>20 belong 164 (Juniper Passive Monitor PIC >20 belong 165 (BACnet MS/TP >20 belong 166 (PPPD +>20 belong 167 (Juniper PPPoE +>20 belong 168 (Juniper PPPoE/ATM >20 belong 169 (GPRS LLC +>20 belong 170 (GPF-T +>20 belong 171 (GPF-F +>20 belong 174 (Juniper PIC Peer +>20 belong 175 (Ethernet with Endace ERF header +>20 belong 176 (Packet-over-SONET with Endace ERF header >20 belong 177 (Linux LAPD +>20 belong 178 (Juniper Ethernet +>20 belong 179 (Juniper PPP +>20 belong 180 (Juniper Frame Relay +>20 belong 181 (Juniper C-HDLC +>20 belong 182 (FRF.16 Frame Relay +>20 belong 183 (Juniper Voice PIC +>20 belong 184 (Arinc 429 +>20 belong 185 (Arinc 653 Interpartition Communication +>20 belong 186 (USB with FreeBSD header >20 belong 187 (Bluetooth HCI H4 +>20 belong 188 (802.16 MAC Common Part Sublayer >20 belong 189 (Linux USB +>20 belong 190 (Controller Area Network (CAN) v. 2.0B +>20 belong 191 (802.15.4 with Linux padding >20 belong 192 (PPI ->20 belong 195 (802.15.4 +>20 belong 193 (802.16 MAC Common Part Sublayer plus radiotap header +>20 belong 194 (Juniper Integrated Service Module +>20 belong 195 (802.15.4 with FCS >20 belong 196 (SITA >20 belong 197 (Endace ERF +>20 belong 198 (Ethernet with u10 Networks pseudo-header +>20 belong 199 (IPMB +>20 belong 200 (Juniper Secure Tunnel >20 belong 201 (Bluetooth HCI H4 with pseudo-header >20 belong 202 (AX.25 with KISS header >20 belong 203 (LAPD @@ -164,7 +200,12 @@ >20 belong 206 (Frame Relay with direction pseudo-header >20 belong 209 (Linux IPMB >20 belong 215 (802.15.4 with non-ASK PHY header +>20 belong 216 (Linux evdev events +>20 belong 219 (MPLS with label as link-layer header >20 belong 220 (Memory-mapped Linux USB +>20 belong 221 (DECT +>20 belong 222 (AOS Space Data Link protocol +>20 belong 223 (Wireless HART >20 belong 224 (Fibre Channel FC-2 >20 belong 225 (Fibre Channel FC-2 with frame delimiters >20 belong 226 (Solaris IPNET @@ -173,69 +214,114 @@ >20 belong 229 (Raw IPv6 >20 belong 230 (802.15.4 without FCS >20 belong 231 (D-Bus messages +>20 belong 232 (Juniper Virtual Server +>20 belong 233 (Juniper SRX E2E +>20 belong 234 (Juniper Fibre Channel >20 belong 235 (DVB-CI >20 belong 236 (MUX27010 >20 belong 237 (STANAG 5066 D_PDUs ->20 belong 239 (Linux netlink NFLOG messages +>20 belong 238 (Juniper ATM CEMIC +>20 belong 239 (Linux netfilter log messages >20 belong 240 (Hilscher netAnalyzer >20 belong 241 (Hilscher netAnalyzer with delimiters >20 belong 242 (IP-over-Infiniband >20 belong 243 (MPEG-2 Transport Stream packets >20 belong 244 (ng4t ng40 >20 belong 245 (NFC LLCP ->20 belong 247 (Infiniband +>20 belong 246 (Packet filter state syncing +>20 belong 247 (InfiniBand >20 belong 248 (SCTP ->16 belong x \b, capture length %d) +>20 belong 249 (USB with USBPcap header +>20 belong 250 (Schweitzer Engineering Laboratories RTAC packets +>20 belong 251 (Bluetooth Low Energy air interface +>20 belong 252 (Wireshark Upper PDU export +>20 belong 253 (Linux netlink +>20 belong 254 (Bluetooth Linux Monitor +>20 belong 255 (Bluetooth Basic Rate/Enhanced Data Rate baseband packets +>20 belong 256 (Bluetooth Low Energy air interface with pseudo-header +>20 belong 257 (PROFIBUS data link layer +>20 belong 258 (Apple DLT_PKTAP +>20 belong 259 (Ethernet with 802.3 Clause 65 EPON preamble +>20 belong 260 (IPMI trace packets +>20 belong 261 (Z-Wave RF profile R1 and R2 packets +>20 belong 262 (Z-Wave RF profile R3 packets +>20 belong 263 (WattStopper Digital Lighting Mngmt/Legrand Nitoo Open Proto +>20 belong 264 (ISO 14443 messages +>20 belong 265 (IEC 62106 Radio Data System groups +>20 belong 266 (USB with Darwin header +>20 belong 267 (OpenBSD DLT_OPENFLOW +>20 belong 268 (IBM SDLC frames +>20 belong 269 (TI LLN sniffer frames +>20 belong 271 (Linux vsock +>20 belong 272 (Nordic Semiconductor Bluetooth LE sniffer frames +>20 belong 273 (Excentis XRA-31 DOCSIS 3.1 RF sniffer frames +>20 belong 274 (802.3br mPackets +>20 belong 275 (DisplayPort AUX channel monitoring data +>20 belong 276 (Linux cooked v2 +>20 belong 278 (OpenVizsla USB +>20 belong 279 (Elektrobit High Speed Capture and Replay (EBHSCR) +>20 belong 281 (Broadcom tag +>20 belong 282 (Broadcom tag (prepended) +# print default match +>20 default x +>>20 belong x (linktype#%u +>16 belong x \b, capture length %u) -0 ubelong 0xa1b2c3d4 tcpdump capture file (big-endian) +# packets time stamps in seconds and microseconds. +0 ubelong 0xa1b2c3d4 pcap capture file, microseconds ts (big-endian) !:mime application/vnd.tcpdump.pcap >0 use pcap-be -0 ulelong 0xa1b2c3d4 tcpdump capture file (little-endian) +0 ulelong 0xa1b2c3d4 pcap capture file, microsecond ts (little-endian) +!:mime application/vnd.tcpdump.pcap +>0 use \^pcap-be + +# packets time stamps in seconds and nanoseconds. +0 ubelong 0xa1b23c4d pcap capture file, nanosecond ts (big-endian) +!:mime application/vnd.tcpdump.pcap +>0 use pcap-be +0 ulelong 0xa1b23c4d pcap capture file, nanosecond ts (little-endian) !:mime application/vnd.tcpdump.pcap >0 use \^pcap-be # # "libpcap"-with-Alexey-Kuznetsov's-patches capture files. -# (We call them "tcpdump capture file(s)" for now, as "tcpdump" is -# the main program that uses that format, but there are other programs -# that use "libpcap", or that use the same capture file format.) # -0 ubelong 0xa1b2cd34 extended tcpdump capture file (big-endian) +0 ubelong 0xa1b2cd34 pcap capture file, microsecond ts, extensions (big-endian) >0 use pcap-be -0 ulelong 0xa1b2cd34 extended tcpdump capture file (little-endian) +0 ulelong 0xa1b2cd34 pcap capture file, microsecond ts, extensions (little-endian) >0 use \^pcap-be # -# "pcap-ng" capture files. -# http://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html -# Pcap-ng files can contain multiple sections. Printing the endianness, +# "pcapng" capture files. +# https://github.com/pcapng/pcapng +# Pcapng files can contain multiple sections. Printing the endianness, # snaplen, or other information from the first SHB may be misleading. # 0 ubelong 0x0a0d0d0a ->8 ubelong 0x1a2b3c4d pcap-ng capture file +>8 ubelong 0x1a2b3c4d pcapng capture file >>12 beshort x - version %d >>14 beshort x \b.%d 0 ulelong 0x0a0d0d0a ->8 ulelong 0x1a2b3c4d pcap-ng capture file +>8 ulelong 0x1a2b3c4d pcapng capture file >>12 leshort x - version %d >>14 leshort x \b.%d # # AIX "iptrace" capture files. # -0 string iptrace\ 1.0 "iptrace" capture file -0 string iptrace\ 2.0 "iptrace" capture file +0 string iptrace\0401.0 AIX iptrace capture file +0 string iptrace\0402.0 AIX iptrace capture file # # Novell LANalyzer capture files. # -0 leshort 0x1001 LANalyzer capture file -0 leshort 0x1007 LANalyzer capture file +0 leshort 0x1001 Novell LANalyzer capture file +0 leshort 0x1007 Novell LANalyzer capture file # # HP-UX "nettl" capture files. # -0 string \x54\x52\x00\x64\x00 "nettl" capture file +0 string \x54\x52\x00\x64\x00 HP/UX nettl capture file # # RADCOM WAN/LAN Analyzer capture files. diff --git a/magic/Magdir/sql b/magic/Magdir/sql index acc452931084..28d89e63bf48 100644 --- a/magic/Magdir/sql +++ b/magic/Magdir/sql @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sql,v 1.21 2017/03/17 21:35:28 christos Exp $ +# $File: sql,v 1.22 2019/04/19 00:42:27 christos Exp $ # sql: file(1) magic for SQL files # # From: "Marty Leisner" <mleisner@eng.mc.xerox.com> @@ -119,13 +119,13 @@ # SQLite Write-Ahead Log from SQLite version >= 3.7.0 -# http://www.sqlite.org/fileformat.html#walformat +# https://www.sqlite.org/fileformat.html#walformat 0 belong&0xfffffffe 0x377f0682 SQLite Write-Ahead Log, !:ext sqlite-wal/db-wal >4 belong x version %d # SQLite Rollback Journal -# http://www.sqlite.org/fileformat.html#rollbackjournal +# https://www.sqlite.org/fileformat.html#rollbackjournal 0 string \xd9\xd5\x05\xf9\x20\xa1\x63\xd7 SQLite Rollback Journal # Panasonic channel list database svl.bin or svl.db added by Joerg Jenderek @@ -137,5 +137,5 @@ #!:mime application/x-panasonic-sqlite3 >>&-15 indirect x \b; contains -# H2 Database from http://www.h2database.com/ +# H2 Database from https://www.h2database.com/ 0 string --\ H2\ 0.5/B\ --\ \n H2 Database file diff --git a/magic/Magdir/sun b/magic/Magdir/sun index 802a9eb8896d..df83834d2dd1 100644 --- a/magic/Magdir/sun +++ b/magic/Magdir/sun @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: sun,v 1.27 2014/04/30 21:41:02 christos Exp $ +# $File: sun,v 1.28 2019/04/19 00:42:27 christos Exp $ # sun: file(1) magic for Sun machines # # Values for big-endian Sun (MC680x0, SPARC) binaries on pre-5.x @@ -93,7 +93,7 @@ 0 string #SUNPC_CONFIG SunPC 4.0 Properties Values # Sun snoop (see RFC 1761, which describes the capture file format, # RFC 3827, which describes some additional datalink types, and -# http://www.iana.org/assignments/snoop-datalink-types/snoop-datalink-types.xml, +# https://www.iana.org/assignments/snoop-datalink-types/snoop-datalink-types.xml, # which is the IANA registry of Snoop datalink types) # 0 string snoop Snoop capture file diff --git a/magic/Magdir/sysex b/magic/Magdir/sysex index fc9cbf4ed4a2..967ac0ce30c1 100644 --- a/magic/Magdir/sysex +++ b/magic/Magdir/sysex @@ -1,11 +1,11 @@ #------------------------------------------------------------------------ -# $File: sysex,v 1.9 2017/03/17 21:35:28 christos Exp $ +# $File: sysex,v 1.10 2019/04/19 00:42:27 christos Exp $ # sysex: file(1) magic for MIDI sysex files # # GRR: original 1 byte test at offset was too general as it catches also many FATs of DOS filesystems # where real SYStem EXclusive messages at offset 1 are limited to seven bits -# http://en.wikipedia.org/wiki/MIDI +# https://en.wikipedia.org/wiki/MIDI 0 ubeshort&0xFF80 0xF000 SysEx File - # North American Group @@ -212,7 +212,7 @@ >1 byte 0x52 Zoom >1 byte 0x54 Matsushita >1 byte 0x57 Acoustic tech. lab. -# http://www.midi.org/techspecs/manid.php +# https://www.midi.org/techspecs/manid.php >1 belong&0xffffff00 0x00007400 Ta Horng >1 belong&0xffffff00 0x00007500 e-Tek >1 belong&0xffffff00 0x00007600 E-Voice diff --git a/magic/Magdir/tcl b/magic/Magdir/tcl index 515fa8dbb9bd..edc3ec42b461 100644 --- a/magic/Magdir/tcl +++ b/magic/Magdir/tcl @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ # file: file(1) magic for Tcl scripting language -# URL: http://www.tcl.tk/ +# URL: https://www.tcl.tk/ # From: gustaf neumann # Tcl scripts diff --git a/magic/Magdir/terminfo b/magic/Magdir/terminfo index c0c4ee254add..fc3bf7458ecd 100644 --- a/magic/Magdir/terminfo +++ b/magic/Magdir/terminfo @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: terminfo,v 1.10 2018/01/21 03:26:33 christos Exp $ +# $File: terminfo,v 1.11 2019/04/19 00:42:27 christos Exp $ # terminfo: file(1) magic for terminfo # -# URL: http://invisible-island.net/ncurses/man/term.5.html -# URL: http://invisible-island.net/ncurses/man/scr_dump.5.html +# URL: https://invisible-island.net/ncurses/man/term.5.html +# URL: https://invisible-island.net/ncurses/man/scr_dump.5.html # # Workaround for Targa image type by Joerg Jenderek # GRR: line below too general as it catches also diff --git a/magic/Magdir/tex b/magic/Magdir/tex index 1737ea95c1d6..aaeae169f336 100644 --- a/magic/Magdir/tex +++ b/magic/Magdir/tex @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: tex,v 1.20 2014/03/16 02:53:03 christos Exp $ +# $File: tex,v 1.21 2019/04/19 00:42:27 christos Exp $ # tex: file(1) magic for TeX files # # XXX - needs byte-endian stuff (big-endian and little-endian DVI?) @@ -108,7 +108,7 @@ 0 string #LyX LyX document text # ConTeXt documents -# http://wiki.contextgarden.net/ +# https://wiki.contextgarden.net/ 0 search/4096 \\setupcolors[ ConTeXt document text !:strength + 15 0 search/4096 \\definecolor[ ConTeXt document text diff --git a/magic/Magdir/tplink b/magic/Magdir/tplink index a3764e2b61e3..6ba6822c757e 100644 --- a/magic/Magdir/tplink +++ b/magic/Magdir/tplink @@ -1,17 +1,19 @@ #------------------------------------------------------------------------------ -# $File: tplink,v 1.2 2017/12/14 05:52:56 christos Exp $ +# $File: tplink,v 1.4 2019/04/19 00:42:27 christos Exp $ # tplink: File magic for openwrt firmware files # URL: https://wiki.openwrt.org/doc/techref/header -# Reference: http://git.openwrt.org/?p=openwrt.git;a=blob;f=tools/firmware-utils/src/mktplinkfw.c +# Reference: https://git.openwrt.org/?p=openwrt.git;a=blob;f=tools/firmware-utils/src/mktplinkfw.c # From: Joerg Jenderek # check for valid header version 1 or 2 0 ulelong <3 >0 ulelong !0 # test for header padding with nulls >>0x100 long 0 ->>>0 use firmware-tplink +# skip Norton Commander Cleanup Utility NCCLEAN.INI by looking for valid vendor +>>>4 ubelong >0x1F000000 +>>>>0 use firmware-tplink 0 name firmware-tplink >0 ubyte x firmware @@ -63,11 +65,13 @@ # look for kernel type (gzip compressed vmlinux.bin by ./compress) >(0x80.L) indirect x # root file system data offset +# WRONG in 5.35 with above indirect expression >0x88 ubelong x \b, at 0x%x # rootfs data length and 1 space >0x8C ubelong x %u bytes # in 5.32 only true for offset ~< FILE_BYTES_MAX=9 MB defined in ../../src/file.h >(0x88.L) indirect x +# 'qshs' for wr940nv1_en_3_13_7_up(111228).bin #>(0x88.L) string x \b, file system '%.4s' #>(0x88.L) ubequad x \b, file system 0x%llx # bootloader data offset diff --git a/magic/Magdir/unicode b/magic/Magdir/unicode index f7eb5a2108aa..7ca61bacbe21 100644 --- a/magic/Magdir/unicode +++ b/magic/Magdir/unicode @@ -1,10 +1,9 @@ #------------------------------------------------------------------------------ -# $File: unicode,v 1.6 2010/09/20 18:55:20 rrt Exp $ +# $File: unicode,v 1.7 2019/02/19 20:34:42 christos Exp $ # Unicode: BOM prefixed text files - Adrian Havill <havill@turbolinux.co.jp> -# GRR: These types should be recognised in file_ascmagic so these -# encodings can be treated by text patterns. -# Missing types are already dealt with internally. +# These types are recognised in file_ascmagic so these encodings can be +# treated by text patterns. Missing types are already dealt with internally. # 0 string +/v8 Unicode text, UTF-7 0 string +/v9 Unicode text, UTF-7 diff --git a/magic/Magdir/varied.script b/magic/Magdir/varied.script index 7f4e405d529c..11e6eb56bb9e 100644 --- a/magic/Magdir/varied.script +++ b/magic/Magdir/varied.script @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: varied.script,v 1.11 2015/03/27 17:59:39 christos Exp $ +# $File: varied.script,v 1.12 2019/04/19 00:42:27 christos Exp $ # varied.script: file(1) magic for various interpreter scripts 0 string/t #!\ / a @@ -45,7 +45,7 @@ # From: arno <arenevier@fdn.fr> # mozilla xpconnect typelib -# see http://www.mozilla.org/scriptable/typelib_file.html +# see https://www.mozilla.org/scriptable/typelib_file.html 0 string XPCOM\nTypeLib\r\n\032 XPConnect Typelib >0x10 byte x version %d >>0x11 byte x \b.%d diff --git a/magic/Magdir/virtual b/magic/Magdir/virtual index 26442bf94b98..347895a96fc9 100644 --- a/magic/Magdir/virtual +++ b/magic/Magdir/virtual @@ -1,14 +1,197 @@ #------------------------------------------------------------------------------ -# $File: virtual,v 1.6 2014/05/07 21:25:41 christos Exp $ +# $File: virtual,v 1.10 2019/04/19 00:42:27 christos Exp $ # From: James Nobis <quel@quelrod.net> # Microsoft hard disk images for: # Virtual Server # Virtual PC -# http://technet.microsoft.com/en-us/virtualserver/bb676673.aspx -# .vhd +# VirtualBox +# URL: http://fileformats.archiveteam.org/wiki/VHD_(Virtual_Hard_Disk) +# Reference: https://download.microsoft.com/download/f/f/e/ffef50a5-07dd-4cf8-aaa3-442c0673a029/ +# Virtual%20Hard%20Disk%20Format%20Spec_10_18_06.doc 0 string conectix Microsoft Disk Image, Virtual Server or Virtual PC +# alternative shorter names +#0 string conectix Microsoft Virtual Hard Disk image +#0 string conectix Microsoft Virtual HD image +!:mime application/x-virtualbox-vhd +!:ext vhd +# Features is a bit field used to indicate specific feature support +#>8 ubelong !0x00000002 \b, Features 0x%x +# Reserved. This bit must always be set to 1. +#>8 ubelong &0x00000002 \b, Reserved 0x%x +# File Format Version for the current specification 0x00010000 +#>12 ubelong !0x00010000 \b, Version 0x%8.8x +# Data Offset only found 0x200 +#>16 ubequad !0x200 \b, Data Offset 0x%llx +#>16 ubequad x \b, at 0x%llx +# Dynamic Disk Header cookie like cxsparse +#>(16.Q) string x "%-.8s" +# This field contains a Unicode string (UTF-16) of the parent hard disk filename +#>(16.Q+64) ubequad x \b, parent name 0x%llx +# Creator Application +# vpc~Microsoft Virtual PC, vs~Microsoft Virtual Server, vbox~VirtualBox, d2v~disk2vhd +>28 string x \b, Creator %-4.4s +# Creator Version: 0x00010000~Virtual Server 2004, 0x00050000~Virtual PC 2004 +# holds the major/minor version of the application that created the image +>32 ubeshort x %x +>34 ubeshort x \b.%x +#>32 ubelong x \b, Version 0x%8.8x +# Creator Host OS: 0x5769326B~Windows (Wi2k), 0x4D616320~Macintosh (Mac) +>36 ubelong x ( +>>36 ubelong 0x5769326B \bW2k +>>36 ubelong 0x4D616320 \bMac +>>36 default x \b0x +>>>36 ubelong x \b%8.8x +# creation Time in seconds since 1 Jan 2000 UTC~946684800 sec. since Unix Epoch +>24 bedate+946684800 x \b) %s +# Original Size +#>40 ubequad x \b, o.-Size 0x%llx +# Current Size is same as original size, but change when disk is expanded +#>48 ubequad x \b, Size 0x%llx +>48 ubequad x \b, %llu bytes +# Disk Geometry: cylinder, heads, and sectors/track for hard disk +#>56 ubeshort x \b, Cylinder 0x%x +>56 ubeshort x \b, CHS %u +# Heads +#>58 ubyte x \b, Heads 0x%x +>58 ubyte x \b/%u +# Sectors per track +#>59 ubyte x \b, Sectors 0x%x +>59 ubyte x \b/%u +# Disk Type: 3~Dynamic hard disk +>60 ubelong !0x3 \b, type 0x%x +# Checksum +#>64 ubelong x \b, cksum 0x%x +# universally unique identifier (UUID) to associate a parent with its differencing image +#>68 ubequad x \b, id 0x%16.16llx +#>76 ubequad x \b-%16.16llx +# Saved State: 1~Saved State +>84 ubyte !0 \b, State 0x%x +# Reserved 427 bytes with nils +#>85 ubequad !0 \b, Reserved 0x%16.16llx +# From: Joerg Jenderek +# URL: https://msdn.microsoft.com/en-us/library/mt740058.aspx +# Reference: https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/ +# MS-VHDX/[MS-VHDX].pdf +# Note: extends the VHD format with new capabilities, such as a 16TB maximum size +# TODO: find and display values like virtual size, disk size, cluster_size, etc +# display id in GUID format +# +# VHDX_FILE_IDENTIFIER signature 0x656C696678646876 +0 string vhdxfile +# VHDX_HEADER signature. 1 header is stored at offset 64KB and the other at 128KB +>0x10000 string head Microsoft Disk Image eXtended +#>0x20000 string head \b, 2nd header +#!:mime application/x-virtualbox-vhdx +!:ext vhdx +# Creator[256] like "QEMU v3.0.0", "Microsoft Windows 6.3.9600.18512" +>>8 lestring16 x \b, by %.256s +# The Checksum field is a CRC-32C hash over the entire 4 KB structure +#>>0x10004 ulelong x \b, CRC 0x%x +# SequenceNumber +>>0x10008 ulequad x \b, sequence 0x%llx +# FileWriteGuid +#>>0x10010 ubequad x \b, file id 0x%llx +#>>>0x10018 ubequad x \b-%llx +# DataWriteGuid +#>>0x10020 ubequad x \b, data id 0x%llx +#>>>0x10028 ubequad x \b-%llx +# LogGuid. If this field is zero, then the log is empty or has no valid entries +>>0x10030 ubequad >0 \b, log id 0x%llx +>>>0x10038 ubequad x \b-%llx +# LogVersion. If not 0 there is a log to replay +>>0x10040 uleshort >0 \b, LogVersion 0x%x +# Version. This field must be set to 1 +>>0x10042 uleshort !1 \b, Version 0x%x +# LogLength must be multiples of 1 MB +>>0x10044 ulelong/1048576 >1 \b, LogLength %u MB +# LogOffset (normally 0x100000 when log direct after header); multiples of 1 MB +>>0x10048 ulequad !0x100000 \b, LogOffset 0x%llx +# Log Entry Signature must be 0x65676F6C~loge +>>(0x10048.q) ulelong !0x65676F6C \b, NO Log Signature +>>(0x10048.q) ulelong =0x65676F6C \b; LOG +# Log Entry Checksum +#>>>(0x10048.q+4) ulelong x \b, Log CRC 0x%x +# Log Entry Length must be a multiple of 4 KB +>>>(0x10048.q+8) ulelong/1024 >4 \b, EntryLength %u KB +# Log Entry Tail must be a multiple of 4 KB +#>>>(0x10048.q+12) ulelong x \b, Tail 0x%x +# Log Entry SequenceNumber +#>>>(0x10048.q+16) ulequad x \b, # 0x%llx +# Log Entry DescriptorCount may be zero. only 4 bytes in other docs instead 8 +#>>>(0x10048.q+24) ulelong x \b, DescriptorCount 0x%llx +# Log Entry Reserved must be set to 0 +>>>(0x10048.q+28) ulelong !0 \b, Reserved 0x%x +# Log Entry LogGuid +#>>>(0x10048.q+32) ubequad x \b, Log id 0x%llx +#>>>(0x10048.q+40) ubequad x \b-%llx +# Log Entry FlushedFileOffset should VHDX size when entry is written. +#>>>(0x10048.q+48) ulequad x \b, FlushedFileOffset %llu +# Log Entry LastFileOffset +#>>>(0x10048.q+56) ulequad x \b, LastFileOffset %llu +# filling +#>>>(0x10048.q+64) ulequad >0 \b, filling %llx +# Reserved[4016] +#>>0x10050 ulequad >0 \b, Reserved 0x%llx +# VHDX_REGION_TABLE_HEADER Signature 0x69676572~regi at offset 192 KB and 256 KB +>0x30000 ulelong !0x69676572 \b, 1st region INVALID +>0x30000 ulelong =0x69676572 \b; region +# region Checksum. CRC-32C hash over the entire 64-KB table +#>>0x30004 ulelong x \b, CRC 0x%x +# The EntryCount specifies number of valid entries; Found 2; This must be =< 2047. +>>0x30008 ulelong x \b, %u entries +# reserved must be zero +#>>0x3000C ulelong !0 \b, RESERVED 0x%x +# Region Table Entry starts with identifier for the object. often BAT id +>>0x30010 use vhdx-id +# FileOffset +>>0x30020 ulequad x \b, at 0x%llx +# Length. Specifies the length of the object within the file +#>>0x30028 ulelong x \b, Length 0x%x +# 1 means region entry is required. if region not recognized, then REFUSE to load VHDX +>>0x3002C ulelong x \b, Required %u +# 2nd region entry often metadata id +>>0x30030 use vhdx-id +# 2nd entry FileOffset +>>0x30040 ulequad x \b, at 0x%llx +# 1 means region entry is required. if region not recognized, then REFUSE to load VHDX +>>0x3004C ulelong x \b, Required %u +# 2nd region +>>0x40000 ulelong !0x69676572 \b, 2nd region INVALID +# check in vhdx images for known id and show names instead hexadecimal +0 name vhdx-id +# https://www.windowstricks.in/online-windows-guid-converter +# 2DC27766-F623-4200-9D64-115E9BFD4A08 BAT GUID +# 6677C22D23F600429D64115E9BFD4A08 BAT ID +>0 ubequad =0x6677C22D23F60042 +>>8 ubequad =0x9D64115E9BFD4A08 \b, id BAT +# no BAT id +>>8 default x +>>>0 use vhdx-id-hex +# 8B7CA206-4790-4B9A-B8FE-575F050F886E Metadata region GUID +# 06A27C8B90479A4BB8FE575F050F886E Metadata region ID +>0 ubequad =0x06A27C8B90479A4B +>>8 ubequad =0xB8FE575F050F886E \b, id Metadata +# no Metadata id +>>8 default x +>>>0 use vhdx-id-hex +# 2FA54224-CD1B-4876-B211-5DBED83BF4B8 Virtual Disk Size GUID +# 2442A52F1BCD7648B2115DBED83BF4B8 Virtual Disk Size ID +# value "virtual size" can be verified by command `qemu-img info ` +>0 ubequad =0x2442A52F1BCD7648 +>>8 ubequad =0xB2115DBED83BF4B8 \b, id vsize +# no Virtual Disk Size ID +>>8 default x +>>>0 use vhdx-id-hex +# other ids +>0 default x +>>0 use vhdx-id-hex +# in vhdx images show id as hexadecimal +0 name vhdx-id-hex +>0 ubequad x \b, ID 0x%16.16llx +>8 ubequad x \b-%16.16llx +# # libvirt # From: Philipp Hahn <hahn@univention.de> 0 string LibvirtQemudSave Libvirt QEMU Suspend Image @@ -36,15 +219,16 @@ # Updated by Adam Buchbinder (adam.buchbinder@gmail.com) # Made by reading sources, reading documentation, and doing trial and error # on existing QCOW files -0 string/b QFI\xFB QEMU QCOW Image +0 string/b QFI\xFB # Uncomment the following line to display Magic (only used for debugging # this magic number) #>0 string/b x , Magic: %s # There are currently 2 Versions: "1" and "2". -# http://www.gnome.org/~markmc/qcow-image-format-version-1.html ->4 belong 1 (v1) +# https://www.gnome.org/~markmc/qcow-image-format-version-1.html +>4 belong !1 QEMU QCOW2 Image +>4 belong 1 QEMU QCOW Image (v1) # Using the existence of the Backing File Offset to determine whether # to read Backing File Information @@ -66,7 +250,7 @@ # 1 for AES encryption, 0 for none. >>36 belong 1 \b, AES-encrypted -# http://www.gnome.org/~markmc/qcow-image-format.html +# https://www.gnome.org/~markmc/qcow-image-format.html >4 belong 2 (v2) # Using the existence of the Backing File Offset to determine whether # to read Backing File Information @@ -102,7 +286,7 @@ 0 string/b QEVM QEMU suspend to disk image # QEMU QED Image -# http://wiki.qemu.org/Features/QED/Specification +# https://wiki.qemu.org/Features/QED/Specification 0 string/b QED\0 QEMU QED Image # VDI Image diff --git a/magic/Magdir/warc b/magic/Magdir/warc index f4ba079a1a9d..5942867ddf62 100644 --- a/magic/Magdir/warc +++ b/magic/Magdir/warc @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: warc,v 1.3 2010/11/25 15:05:43 christos Exp $ +# $File: warc,v 1.4 2019/04/19 00:42:27 christos Exp $ # warc: file(1) magic for WARC files 0 string WARC/ WARC Archive @@ -9,7 +9,7 @@ #------------------------------------------------------------------------------ # Arc File Format from Internet Archive -# see http://www.archive.org/web/researcher/ArcFileFormat.php +# see https://www.archive.org/web/researcher/ArcFileFormat.php 0 string filedesc:// Internet Archive File !:mime application/x-ia-arc >11 search/256 \x0A \b diff --git a/magic/Magdir/webassembly b/magic/Magdir/webassembly index 8a4ce1b1a599..3b1d37e667bd 100644 --- a/magic/Magdir/webassembly +++ b/magic/Magdir/webassembly @@ -1,12 +1,12 @@ #------------------------------------------------------------------------------ -# $File: webassembly,v 1.2 2017/05/02 14:05:29 christos Exp $ +# $File: webassembly,v 1.3 2019/04/19 00:42:27 christos Exp $ # webassembly: file(1) magic for WebAssembly modules # # WebAssembly is a virtual architecture developed by a W3C Community -# Group at http://webassembly.org/. The file extension is .wasm, and +# Group at https://webassembly.org/. The file extension is .wasm, and # the MIME type is application/wasm. # -# http://webassembly.org/docs/binary-encoding/ is the main +# https://webassembly.org/docs/binary-encoding/ is the main # document describing the binary format. # From: Pip Cet <pipcet@gmail.com> and Joel Martin diff --git a/magic/Magdir/windows b/magic/Magdir/windows index f8a9c83d5ee7..39ed3e2bec15 100644 --- a/magic/Magdir/windows +++ b/magic/Magdir/windows @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: windows,v 1.22 2018/02/16 15:44:00 christos Exp $ +# $File: windows,v 1.26 2019/05/01 17:55:25 christos Exp $ # windows: file(1) magic for Microsoft Windows # # This file is mainly reserved for files where programs @@ -26,8 +26,8 @@ # Summary: Windows crash dump # Extension: .dmp -# Created by: Andreas Schuster (http://computer.forensikblog.de/) -# Reference (1): http://computer.forensikblog.de/en/2008/02/64bit_magic.html +# Created by: Andreas Schuster (https://computer.forensikblog.de/) +# Reference (1): https://computer.forensikblog.de/en/2008/02/64bit_magic.html # Modified by (1): Abel Cheung (Avoid match with first 4 bytes only) 0 string PAGE >4 string DUMP MS Windows 32bit crash dump @@ -46,8 +46,8 @@ # Summary: Vista Event Log # Extension: .evtx -# Created by: Andreas Schuster (http://computer.forensikblog.de/) -# Reference (1): http://computer.forensikblog.de/en/2007/05/some_magic.html +# Created by: Andreas Schuster (https://computer.forensikblog.de/) +# Reference (1): https://computer.forensikblog.de/en/2007/05/some_magic.html 0 string ElfFile\0 MS Windows Vista Event Log >0x2a leshort x \b, %d chunks >>0x10 lelong x \b (no. %d in use) @@ -56,6 +56,80 @@ >0x78 lelong &1 \b, DIRTY >0x78 lelong &2 \b, FULL +# Summary: Windows System Deployment Image +# Created by: Joerg Jenderek +# URL: http://en.wikipedia.org/wiki/System_Deployment_Image +# Reference: http://skolk.livejournal.com/1320.html +0 string $SDI +>4 string 0001 System Deployment Image +!:mime application/x-ms-sdi +#!:mime application/octet-stream +# \Boot\boot.sdi +!:ext sdi +# MDBtype: 0~Unspecified 1~RAM 2~ROM +>>8 ulequad !0 \b, MDBtype 0x%llx +# BootCodeOffset +>>16 ulequad !0 \b, BootCodeOffset 0x%llx +# BootCodeSize +>>24 ulequad !0 \b, BootCodeSize 0x%llx +# VendorID +>>32 ulequad !0 \b, VendorID 0x%llx +# DeviceID +>>40 ulequad !0 \b, DeviceID 0x%llx +# DeviceModel +>>48 ulequad !0 \b, DeviceModel 0x%llx +>>>56 ulequad !0 \b%llx +# DeviceRole +>>64 ulequad !0 \b, DeviceRole 0x%llx +# Reserved1; reserved fields and gaps between BLOBs are padded with \0 +#>>72 ulequad !0 \b, Reserved1 0x%llx +# RuntimeGUID +>>80 ulequad !0 \b, RuntimeGUID 0x%llx +>>>88 ulequad !0 \b%llx +# RuntimeOEMrev +>>96 ulequad !0 \b, RuntimeOEMrev 0x%llx +# Reserved2 +#>>104 ulequad !0 \b, Reserved2 0x%llx +# BLOB alignment value in pages, as specified in sdimgr /pack: 1~4K 2~8k +>>112 ulequad !0 \b, PageAlignment %llu +# Reserved3[48] +#>>120 ulequad !0 \b, Reserved3 0x%llx +# SDI checksum 39h +>>0x1f8 ulequad x \b, checksum 0x%llx +# BLOBtype[8] \0-padded: PART, WIM , BOOT, LOAD, DISK +>>0x400 string >\0 \b, type %-3.8s +# 0~non-filesystem 7~NTFS 6~BIGFAT +>>>0x420 ulequad !0 (0x%llx) +# ATTRibutes +>>>0x408 ulequad !0 0x%llx attributes +# Offset +>>>0x410 ulequad x at 0x%llx +# print 1 space after size and then handles NTFS boot sector by ./filesystems +>>>0x418 ulequad >0 %llu bytes +>>>>(0x410.l) indirect x +# 2nd BLOB: WIM +>>0x440 string >\0 \b, type %-3.8s +>>>0x428 ulequad !0 (0x%llx) +# ATTRibutes +>>>0x448 ulequad !0 0x%llx attributes +# Offset +>>>0x450 ulequad x at 0x%llx +>>>0x458 ulequad >0 %llu bytes +>>>>(0x450.l) indirect x +# 3rd BLOB +>>0x480 string >\0 \b, type %-3.8s + +# Summary: Windows Error Report text files +# URL: https://en.wikipedia.org/wiki/Windows_Error_Reporting +# Reference: https://www.nirsoft.net/utils/app_crash_view.html +# Created by: Joerg Jenderek +# Note: in directories %ProgramData%\Microsoft\Windows\WER\{ReportArchive,ReportQueue} +# %LOCALAPPDATA%\Microsoft\Windows\WER\{ReportArchive,ReportQueue} +0 lestring16 Version= +>22 lestring16 EventType Windows Error Report +!:mime text/plain +# Report.wer +!:ext wer # Summary: Windows 3.1 group files # Extension: .grp @@ -65,7 +139,7 @@ # Summary: Old format help files # URL: https://en.wikipedia.org/wiki/WinHelp -# Reference: http://www.oocities.org/mwinterhoff/helpfile.htm +# Reference: https://www.oocities.org/mwinterhoff/helpfile.htm # Update: Joerg Jenderek # Created by: Dirk Jagdmann <doj@cubic.org> # @@ -213,7 +287,7 @@ 0 string HyperTerminal\040 >15 string 1.0\ --\ HyperTerminal\ data\ file MS Windows HyperTerminal profile -# http://ithreats.files.wordpress.com/2009/05/\040 +# https://ithreats.files.wordpress.com/2009/05/\040 # lnk_the_windows_shortcut_file_format.pdf # Summary: Windows shortcut # Extension: .lnk @@ -343,8 +417,8 @@ 0 name ini-file # look for left bracket in section line >0 search/8192 [ -# http://en.wikipedia.org/wiki/Autorun.inf -# http://msdn.microsoft.com/en-us/library/windows/desktop/cc144200.aspx +# https://en.wikipedia.org/wiki/Autorun.inf +# https://msdn.microsoft.com/en-us/library/windows/desktop/cc144200.aspx # space after right bracket # or AutoRun.Amd64 for 64 bit systems # or only NL separator @@ -360,7 +434,7 @@ >>>&0 string !]\r\n[ Microsoft Windows Autorun file !:mime application/x-setupscript !:ext inf -# http://msdn.microsoft.com/en-us/library/windows/hardware/ff549520(v=vs.85).aspx +# https://msdn.microsoft.com/en-us/library/windows/hardware/ff549520(v=vs.85).aspx # version strings ASCII coded case-independent for Windows setup information script file >>&0 regex/c \^(version|strings)] Windows setup INFormation !:mime application/x-setupscript @@ -371,24 +445,24 @@ !:mime application/x-setupscript !:ext inf # http://www.winfaq.de/faq_html/Content/tip2500/onlinefaq.php?h=tip2653.htm -# http://msdn.microsoft.com/en-us/library/windows/desktop/cc144102.aspx +# https://msdn.microsoft.com/en-us/library/windows/desktop/cc144102.aspx # .ShellClassInfo DeleteOnCopy LocalizedFileNames ASCII coded case-independent >>&0 regex/c \^(\.ShellClassInfo|DeleteOnCopy|LocalizedFileNames)] Windows desktop.ini !:mime application/x-wine-extension-ini #!:mime text/plain -# http://support.microsoft.com/kb/84709/ +# https://support.microsoft.com/kb/84709/ >>&0 regex/c \^(don't\ load)] Windows CONTROL.INI !:mime application/x-wine-extension-ini !:ext ini >>&0 regex/c \^(ndishlp\\$|protman\\$|NETBEUI\\$)] Windows PROTOCOL.INI !:mime application/x-wine-extension-ini !:ext ini -# http://technet.microsoft.com/en-us/library/cc722567.aspx +# https://technet.microsoft.com/en-us/library/cc722567.aspx # http://www.winfaq.de/faq_html/Content/tip0000/onlinefaq.php?h=tip0137.htm >>&0 regex/c \^(windows|Compatibility|embedding)] Windows WIN.INI !:mime application/x-wine-extension-ini !:ext ini -# http://en.wikipedia.org/wiki/SYSTEM.INI +# https://en.wikipedia.org/wiki/SYSTEM.INI >>&0 regex/c \^(boot|386enh|drivers)] Windows SYSTEM.INI !:mime application/x-wine-extension-ini !:ext ini @@ -396,18 +470,18 @@ >>&0 regex/c \^(SafeList)] Windows IOS.INI !:mime application/x-wine-extension-ini !:ext ini -# http://en.wikipedia.org/wiki/NTLDR Windows Boot Loader information +# https://en.wikipedia.org/wiki/NTLDR Windows Boot Loader information >>&0 regex/c \^(boot\x20loader)] Windows boot.ini !:mime application/x-wine-extension-ini !:ext ini -# http://en.wikipedia.org/wiki/CONFIG.SYS +# https://en.wikipedia.org/wiki/CONFIG.SYS >>&0 regex/c \^(menu)] MS-DOS CONFIG.SYS # @CONFIG.UI configuration file of previous DOS version saved by Caldera OPENDOS INSTALL.EXE # CONFIG.PSS saved version of file CONFIG.SYS created by %WINDIR%\SYTEM\MSCONFIG.EXE # CONFIG.TSH renamed file CONFIG.SYS.BAT by %WINDIR%\SYTEM\MSCONFIG.EXE # dos and w40 used in dual booting scene !:ext sys/dos/w40 -# http://support.microsoft.com/kb/118579/ +# https://support.microsoft.com/kb/118579/ >>&0 regex/c \^(Paths)]\r\n MS-DOS MSDOS.SYS !:ext sys/dos # http://chmspec.nongnu.org/latest/INI.html#HHP @@ -423,7 +497,7 @@ >>>>&0 string/c version Windows setup INFormation !:mime application/x-setupscript !:ext inf -# http://en.wikipedia.org/wiki/Initialization_file Windows Initialization File or other +# https://en.wikipedia.org/wiki/Initialization_file Windows Initialization File or other >>>>&0 default x >>>>>&0 ubyte x # characters, digits, underscore and white space followed by right bracket @@ -560,7 +634,7 @@ # Summary: backup file created with utility like NTBACKUP.EXE shipped with Windows NT/2K/XP/2003 # Extension: .bkf # Created by: Joerg Jenderek -# URL: http://en.wikipedia.org/wiki/NTBackup +# URL: https://en.wikipedia.org/wiki/NTBackup # Reference: http://laytongraphics.com/mtf/MTF_100a.PDF # Descriptor BloCK name of Microsoft Tape Format 0 string TAPE @@ -657,7 +731,7 @@ # # URL: https://en.wikipedia.org/wiki/PaintShop_Pro -# Reference: http://www.cryer.co.uk/file-types/p/pal.htm +# Reference: https://www.cryer.co.uk/file-types/p/pal.htm # Created by: Joerg Jenderek # Note: there exist other color palette formats also with .pal extension 0 string JASC-PAL\r\n PaintShop Pro color palette @@ -669,7 +743,7 @@ # third line contains the number of colours: 16 256 ... >16 string x \b, %.3s colors -# URL: http://en.wikipedia.org/wiki/Innosetup +# URL: https://en.wikipedia.org/wiki/Innosetup # Reference: https://github.com/jrsoftware/issrc/blob/master/Projects/Undo.pas # Created by: Joerg Jenderek # Note: created by like "InnoSetup self-extracting archive" inside ./msdos @@ -716,3 +790,92 @@ # directory like C:\Program Files\GIMP 2 >>>>&0 lestring16 x \b, %-.42s +# Windows Imaging (WIM) Image +# Update: Joerg Jenderek at Mar 2019 +# URL: https://en.wikipedia.org/wiki/Windows_Imaging_Format +# Reference: https://download.microsoft.com/download/f/e/f/ +# fefdc36e-392d-4678-9e4e-771ffa2692ab/Windows%20Imaging%20File%20Format.rtf +# Note: verified by like `7z t boot.wim` `wiminfo install.esd --header` +0 string MSWIM\000\000\000 +>0 use wim-archive +# https://wimlib.net/man1/wimoptimize.html +0 string WLPWM\000\000\000 +>0 use wim-archive +0 name wim-archive +# _WIMHEADER_V1_PACKED ImageTag[8] +>0 string x Windows imaging +!:mime application/x-ms-wim +# TO avoid in file version 5.36 error like +# Magdir/windows, 760: Warning: Current entry does not yet have a description +# file: could not find any valid magic files! (No error) +# splitted WIM +>16 ulelong &0x00000008 (SWM +!:ext swm +# usPartNumber; 1, unless the file was split into multiple parts +>>40 uleshort x \b %u +# usTotalParts; The total number of WIM file parts in a spanned set +>>42 uleshort x \b of %u) image +# non splitted WIM +>16 ulelong ^0x00000008 +# https://wimlib.net/man1/wimmount.html +# solid WIMs; version 3584; usually contain LZMS-compressed and the .esd extension +>>12 ulelong 3584 (ESD) image +!:ext esd +>>12 ulelong !3584 (WIM) image +!:ext wim +>0 string/b WLPWM\000\000\000 \b, wimlib pipable format +# cbSize size of the WIM header in bytes like 208 +#>8 ulelong x \b, headersize %u +# dwVersion version of the WIM file 00010d00h~1.13 00000e00h~0.14 +>14 uleshort x v%u +>13 ubyte x \b.%u +# dwImageCount; The number of images contained in the WIM file +>44 ulelong >1 \b, %u images +# dwBootIndex +# 1-based index of the bootable image of the WIM, or 0 if no image is bootable +>0x78 ulelong >0 \b, bootable no. %u +# dwFlags +#>16 ulelong x \b, flags 0x%8.8x +#define FLAG_HEADER_COMPRESSION 0x00000002 +#define FLAG_HEADER_READONLY 0x00000004 +#define FLAG_HEADER_SPANNED 0x00000008 +#define FLAG_HEADER_RESOURCE_ONLY 0x00000010 +#define FLAG_HEADER_METADATA_ONLY 0x00000020 +#define FLAG_HEADER_WRITE_IN_PROGRESS 0x00000040 +#define FLAG_HEADER_RP_FIX 0x00000080 reparse point fixup +#define FLAG_HEADER_COMPRESS_RESERVED 0x00010000 +#define FLAG_HEADER_COMPRESS_XPRESS 0x00020000 +#define FLAG_HEADER_COMPRESS_LZX 0x00040000 +#define FLAG_HEADER_COMPRESS_LZMS 0x00080000 +#define FLAG_HEADER_COMPRESS_XPRESS2 0x00100000 wimlib-1.13.0\include\wimlib\header.h +# XPRESS, with small chunk size +>16 ulelong &0x00100000 \b, XPRESS2 +>16 ulelong &0x00080000 \b, LZMS +>16 ulelong &0x00040000 \b, LZX +>16 ulelong &0x00020000 \b, XPRESS +>16 ulelong &0x00000002 compressed +>16 ulelong &0x00000004 \b, read only +>16 ulelong &0x00000010 \b, resource only +>16 ulelong &0x00000020 \b, metadata only +>16 ulelong &0x00000080 \b, reparse point fixup +#>16 ulelong &0x00010000 \b, RESERVED +# dwCompressionSize; Uncompressed chunk size for resources or 0 if uncompressed +#>20 ulelong >0 \b, chunk size %u bytes +# gWIMGuid +#>24 ubequad x \b, GUID 0x%16.16llx +#>>32 ubequad x \b%16.16llx +# rhOffsetTable; the location of the resource lookup table +# wim_reshdr_disk[24]= u8 size_in_wim[7] + u8 flags + le64 offset_in_wim + le64 uncompressed_size +#>48 ubequad x \b, rhOffsetTable 0x%16.16llx +# rhXmlData; the location of the XML data +#>0x50 ulelong x \b, at 0x%8.8x +# NOT WORKING \xff\xfe<\0W\0I\0M\0 +#>(0x50.l) ubequad x \b, xml=%16.16llx +# rhBootMetadata; the location of the metadata resource +#>0x60 ubequad x \b, rhBootMetadata 0x%16.16llx +# rhIntegrity; the location of integrity table used to verify files +#>0x7c ubequad x \b, rhIntegrity 0x%16.16llx +# Unused[60] +#>148 ubequad !0 \b,unused 0x%16.16llx +# + diff --git a/magic/Magdir/wordprocessors b/magic/Magdir/wordprocessors index a7cfab7c6ca6..4b08c4303102 100644 --- a/magic/Magdir/wordprocessors +++ b/magic/Magdir/wordprocessors @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: wordprocessors,v 1.19 2015/10/16 15:11:07 christos Exp $ +# $File: wordprocessors,v 1.20 2019/04/19 00:42:27 christos Exp $ # wordprocessors: file(1) magic fo word processors. # ####### PWP file format used on Smith Corona Personal Word Processors: @@ -210,7 +210,7 @@ >5 string >\0 version %s 0 string \\1cw ChiWriter file -# Quark Express from http://www.garykessler.net/library/file_sigs.html +# Quark Express from https://www.garykessler.net/library/file_sigs.html 2 string IIXPR3 Intel Quark Express Document (English) 2 string IIXPRa Intel Quark Express Document (Korean) 2 string MMXPR3 Motorola Quark Express Document (English) diff --git a/magic/Magdir/wsdl b/magic/Magdir/wsdl index 94bd1d1110fe..35edafc2f535 100644 --- a/magic/Magdir/wsdl +++ b/magic/Magdir/wsdl @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: wsdl,v 1.3 2013/02/06 14:18:52 christos Exp $ -# wsdl: PHP WSDL Cache, http://www.php.net/manual/en/book.soap.php +# $File: wsdl,v 1.5 2019/04/19 00:42:27 christos Exp $ +# wsdl: PHP WSDL Cache, https://www.php.net/manual/en/book.soap.php # Cache format extracted from source: -# http://svn.php.net/viewvc/php/php-src/trunk/ext/soap/php_sdl.c?revision=HEAD&view=markup -# Requires file >= 5.05, see http://mx.gw.com/pipermail/file/2010/000683.html +# https://svn.php.net/viewvc/php/php-src/trunk/ext/soap/php_sdl.c?revision=HEAD&view=markup +# Requires file >= 5.05 # By Elan Ruusamae <glen@delfi.ee>, Patryk Zawadzki <patrys@pld-linux.org>, 2010-2011 0 string wsdl PHP WSDL cache, >4 byte x version 0x%02x diff --git a/magic/Magdir/xwindows b/magic/Magdir/xwindows index 85f64c8c36a4..7118cadd05ef 100644 --- a/magic/Magdir/xwindows +++ b/magic/Magdir/xwindows @@ -1,6 +1,6 @@ #------------------------------------------------------------------------------ -# $File: xwindows,v 1.10 2017/03/17 21:35:28 christos Exp $ +# $File: xwindows,v 1.11 2019/04/19 00:42:27 christos Exp $ # xwindows: file(1) magic for various X/Window system file formats. # Compiled X Keymap @@ -27,8 +27,8 @@ # Xcursor data # X11 mouse cursor format defined in libXcursor, see -# http://www.x.org/archive/X11R6.8.1/doc/Xcursor.3.html -# http://cgit.freedesktop.org/xorg/lib/libXcursor/tree/include/X11/Xcursor/Xcursor.h +# https://www.x.org/archive/X11R6.8.1/doc/Xcursor.3.html +# https://cgit.freedesktop.org/xorg/lib/libXcursor/tree/include/X11/Xcursor/Xcursor.h 0 string Xcur Xcursor data !:mime image/x-xcursor >10 leshort x version %d diff --git a/magic/Magdir/yara b/magic/Magdir/yara index 523a2d05f410..e581c433a56a 100644 --- a/magic/Magdir/yara +++ b/magic/Magdir/yara @@ -1,8 +1,8 @@ #------------------------------------------------------------------------------ -# $File: yara,v 1.2 2017/05/25 20:07:23 christos Exp $ -# yara: file(1) magic for http://virustotal.github.io/yara/ +# $File: yara,v 1.3 2019/04/19 00:42:27 christos Exp $ +# yara: file(1) magic for https://virustotal.github.io/yara/ # 0 string YARA diff --git a/magic/Magdir/zip b/magic/Magdir/zip index 17919a3d8b10..f214ad0df550 100644 --- a/magic/Magdir/zip +++ b/magic/Magdir/zip @@ -1,12 +1,12 @@ #------------------------------------------------------------------------------ -# $File: zip,v 1.1 2017/11/03 23:36:17 christos Exp $ +# $File: zip,v 1.2 2019/04/09 18:34:15 christos Exp $ # zip: file(1) magic for zip files; this is not use # Note the version of magic in archive is currently stronger, this is # just an example until negative offsets are supported better # Zip Central Cirectory record 0 name zipcd ->0 string PK\001\002 +>0 string PK\001\002 Zip archive data >>4 leshort x \b, made by >>4 use zipversion >>6 leshort x \b, extract using at least @@ -53,7 +53,7 @@ >>0 leshort x v?[%#x] # Zip End Of Central Directory record --22 string PK\005\006 Zip archive data +-22 string PK\005\006 #>4 leshort >1 \b, %d disks #>6 leshort >1 \b, central directory disk %d #>8 leshort >1 \b, %d central directories on this disk diff --git a/magic/Makefile.am b/magic/Makefile.am index 69c9decc5ae5..a52f522c924d 100644 --- a/magic/Makefile.am +++ b/magic/Makefile.am @@ -1,5 +1,5 @@ # -# $File: Makefile.am,v 1.133 2018/06/06 01:16:40 christos Exp $ +# $File: Makefile.am,v 1.143 2019/05/09 16:24:36 christos Exp $ # MAGIC_FRAGMENT_BASE = Magdir MAGIC_DIR = $(top_srcdir)/magic @@ -33,11 +33,12 @@ $(MAGIC_FRAGMENT_DIR)/asterix \ $(MAGIC_FRAGMENT_DIR)/att3b \ $(MAGIC_FRAGMENT_DIR)/audio \ $(MAGIC_FRAGMENT_DIR)/basis \ -$(MAGIC_FRAGMENT_DIR)/ber \ $(MAGIC_FRAGMENT_DIR)/beetle \ +$(MAGIC_FRAGMENT_DIR)/ber \ $(MAGIC_FRAGMENT_DIR)/bflt \ $(MAGIC_FRAGMENT_DIR)/bhl \ $(MAGIC_FRAGMENT_DIR)/bioinformatics \ +$(MAGIC_FRAGMENT_DIR)/biosig \ $(MAGIC_FRAGMENT_DIR)/blackberry \ $(MAGIC_FRAGMENT_DIR)/blcr \ $(MAGIC_FRAGMENT_DIR)/blender \ @@ -58,6 +59,7 @@ $(MAGIC_FRAGMENT_DIR)/citrus \ $(MAGIC_FRAGMENT_DIR)/clarion \ $(MAGIC_FRAGMENT_DIR)/claris \ $(MAGIC_FRAGMENT_DIR)/clipper \ +$(MAGIC_FRAGMENT_DIR)/clojure \ $(MAGIC_FRAGMENT_DIR)/coff \ $(MAGIC_FRAGMENT_DIR)/commands \ $(MAGIC_FRAGMENT_DIR)/communications \ @@ -82,12 +84,14 @@ $(MAGIC_FRAGMENT_DIR)/dolby \ $(MAGIC_FRAGMENT_DIR)/dump \ $(MAGIC_FRAGMENT_DIR)/dyadic \ $(MAGIC_FRAGMENT_DIR)/ebml \ +$(MAGIC_FRAGMENT_DIR)/edid \ $(MAGIC_FRAGMENT_DIR)/editors \ $(MAGIC_FRAGMENT_DIR)/efi \ $(MAGIC_FRAGMENT_DIR)/elf \ $(MAGIC_FRAGMENT_DIR)/encore \ $(MAGIC_FRAGMENT_DIR)/epoc \ $(MAGIC_FRAGMENT_DIR)/erlang \ +$(MAGIC_FRAGMENT_DIR)/espressif \ $(MAGIC_FRAGMENT_DIR)/esri \ $(MAGIC_FRAGMENT_DIR)/fcs \ $(MAGIC_FRAGMENT_DIR)/filesystems \ @@ -106,6 +110,7 @@ $(MAGIC_FRAGMENT_DIR)/gconv \ $(MAGIC_FRAGMENT_DIR)/geo \ $(MAGIC_FRAGMENT_DIR)/geos \ $(MAGIC_FRAGMENT_DIR)/gimp \ +$(MAGIC_FRAGMENT_DIR)/glibc \ $(MAGIC_FRAGMENT_DIR)/gnome \ $(MAGIC_FRAGMENT_DIR)/gnu \ $(MAGIC_FRAGMENT_DIR)/gnumeric \ @@ -115,6 +120,7 @@ $(MAGIC_FRAGMENT_DIR)/grace \ $(MAGIC_FRAGMENT_DIR)/graphviz \ $(MAGIC_FRAGMENT_DIR)/gringotts \ $(MAGIC_FRAGMENT_DIR)/guile \ +$(MAGIC_FRAGMENT_DIR)/hardware \ $(MAGIC_FRAGMENT_DIR)/hitachi-sh \ $(MAGIC_FRAGMENT_DIR)/hp \ $(MAGIC_FRAGMENT_DIR)/human68k \ @@ -136,6 +142,7 @@ $(MAGIC_FRAGMENT_DIR)/karma \ $(MAGIC_FRAGMENT_DIR)/kde \ $(MAGIC_FRAGMENT_DIR)/keepass \ $(MAGIC_FRAGMENT_DIR)/kerberos \ +$(MAGIC_FRAGMENT_DIR)/kicad \ $(MAGIC_FRAGMENT_DIR)/kml \ $(MAGIC_FRAGMENT_DIR)/lecter \ $(MAGIC_FRAGMENT_DIR)/lex \ @@ -147,8 +154,8 @@ $(MAGIC_FRAGMENT_DIR)/lua \ $(MAGIC_FRAGMENT_DIR)/luks \ $(MAGIC_FRAGMENT_DIR)/m4 \ $(MAGIC_FRAGMENT_DIR)/mach \ -$(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/macintosh \ +$(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/magic \ $(MAGIC_FRAGMENT_DIR)/mail.news \ $(MAGIC_FRAGMENT_DIR)/make \ @@ -176,8 +183,8 @@ $(MAGIC_FRAGMENT_DIR)/motorola \ $(MAGIC_FRAGMENT_DIR)/mozilla \ $(MAGIC_FRAGMENT_DIR)/msdos \ $(MAGIC_FRAGMENT_DIR)/msooxml \ -$(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/msvc \ +$(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/mup \ $(MAGIC_FRAGMENT_DIR)/music \ $(MAGIC_FRAGMENT_DIR)/nasa \ @@ -189,6 +196,7 @@ $(MAGIC_FRAGMENT_DIR)/netscape \ $(MAGIC_FRAGMENT_DIR)/netware \ $(MAGIC_FRAGMENT_DIR)/news \ $(MAGIC_FRAGMENT_DIR)/nitpicker \ +$(MAGIC_FRAGMENT_DIR)/numpy \ $(MAGIC_FRAGMENT_DIR)/oasis \ $(MAGIC_FRAGMENT_DIR)/ocaml \ $(MAGIC_FRAGMENT_DIR)/octave \ @@ -228,6 +236,7 @@ $(MAGIC_FRAGMENT_DIR)/revision \ $(MAGIC_FRAGMENT_DIR)/riff \ $(MAGIC_FRAGMENT_DIR)/rpi \ $(MAGIC_FRAGMENT_DIR)/rpm \ +$(MAGIC_FRAGMENT_DIR)/rpmsg \ $(MAGIC_FRAGMENT_DIR)/rtf \ $(MAGIC_FRAGMENT_DIR)/ruby \ $(MAGIC_FRAGMENT_DIR)/sc \ @@ -271,8 +280,8 @@ $(MAGIC_FRAGMENT_DIR)/unicode \ $(MAGIC_FRAGMENT_DIR)/unknown \ $(MAGIC_FRAGMENT_DIR)/uterus \ $(MAGIC_FRAGMENT_DIR)/uuencode \ -$(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/vacuum-cleaner \ +$(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/varied.script \ $(MAGIC_FRAGMENT_DIR)/vax \ $(MAGIC_FRAGMENT_DIR)/vicar \ @@ -300,7 +309,7 @@ $(MAGIC_FRAGMENT_DIR)/yara \ $(MAGIC_FRAGMENT_DIR)/zfs \ $(MAGIC_FRAGMENT_DIR)/zilog \ $(MAGIC_FRAGMENT_DIR)/zip \ -$(MAGIC_FRAGMENT_DIR)/zyxel +$(MAGIC_FRAGMENT_DIR)/zyxel MAGIC = magic.mgc CLEANFILES = ${MAGIC} $(MAGIC_FRAGMENT_DIR)/Localstuff diff --git a/magic/Makefile.in b/magic/Makefile.in index 31490fd1c793..bacc6d575ace 100644 --- a/magic/Makefile.in +++ b/magic/Makefile.in @@ -273,7 +273,7 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ # -# $File: Makefile.am,v 1.133 2018/06/06 01:16:40 christos Exp $ +# $File: Makefile.am,v 1.143 2019/05/09 16:24:36 christos Exp $ # MAGIC_FRAGMENT_BASE = Magdir MAGIC_DIR = $(top_srcdir)/magic @@ -305,11 +305,12 @@ $(MAGIC_FRAGMENT_DIR)/asterix \ $(MAGIC_FRAGMENT_DIR)/att3b \ $(MAGIC_FRAGMENT_DIR)/audio \ $(MAGIC_FRAGMENT_DIR)/basis \ -$(MAGIC_FRAGMENT_DIR)/ber \ $(MAGIC_FRAGMENT_DIR)/beetle \ +$(MAGIC_FRAGMENT_DIR)/ber \ $(MAGIC_FRAGMENT_DIR)/bflt \ $(MAGIC_FRAGMENT_DIR)/bhl \ $(MAGIC_FRAGMENT_DIR)/bioinformatics \ +$(MAGIC_FRAGMENT_DIR)/biosig \ $(MAGIC_FRAGMENT_DIR)/blackberry \ $(MAGIC_FRAGMENT_DIR)/blcr \ $(MAGIC_FRAGMENT_DIR)/blender \ @@ -330,6 +331,7 @@ $(MAGIC_FRAGMENT_DIR)/citrus \ $(MAGIC_FRAGMENT_DIR)/clarion \ $(MAGIC_FRAGMENT_DIR)/claris \ $(MAGIC_FRAGMENT_DIR)/clipper \ +$(MAGIC_FRAGMENT_DIR)/clojure \ $(MAGIC_FRAGMENT_DIR)/coff \ $(MAGIC_FRAGMENT_DIR)/commands \ $(MAGIC_FRAGMENT_DIR)/communications \ @@ -354,12 +356,14 @@ $(MAGIC_FRAGMENT_DIR)/dolby \ $(MAGIC_FRAGMENT_DIR)/dump \ $(MAGIC_FRAGMENT_DIR)/dyadic \ $(MAGIC_FRAGMENT_DIR)/ebml \ +$(MAGIC_FRAGMENT_DIR)/edid \ $(MAGIC_FRAGMENT_DIR)/editors \ $(MAGIC_FRAGMENT_DIR)/efi \ $(MAGIC_FRAGMENT_DIR)/elf \ $(MAGIC_FRAGMENT_DIR)/encore \ $(MAGIC_FRAGMENT_DIR)/epoc \ $(MAGIC_FRAGMENT_DIR)/erlang \ +$(MAGIC_FRAGMENT_DIR)/espressif \ $(MAGIC_FRAGMENT_DIR)/esri \ $(MAGIC_FRAGMENT_DIR)/fcs \ $(MAGIC_FRAGMENT_DIR)/filesystems \ @@ -378,6 +382,7 @@ $(MAGIC_FRAGMENT_DIR)/gconv \ $(MAGIC_FRAGMENT_DIR)/geo \ $(MAGIC_FRAGMENT_DIR)/geos \ $(MAGIC_FRAGMENT_DIR)/gimp \ +$(MAGIC_FRAGMENT_DIR)/glibc \ $(MAGIC_FRAGMENT_DIR)/gnome \ $(MAGIC_FRAGMENT_DIR)/gnu \ $(MAGIC_FRAGMENT_DIR)/gnumeric \ @@ -387,6 +392,7 @@ $(MAGIC_FRAGMENT_DIR)/grace \ $(MAGIC_FRAGMENT_DIR)/graphviz \ $(MAGIC_FRAGMENT_DIR)/gringotts \ $(MAGIC_FRAGMENT_DIR)/guile \ +$(MAGIC_FRAGMENT_DIR)/hardware \ $(MAGIC_FRAGMENT_DIR)/hitachi-sh \ $(MAGIC_FRAGMENT_DIR)/hp \ $(MAGIC_FRAGMENT_DIR)/human68k \ @@ -408,6 +414,7 @@ $(MAGIC_FRAGMENT_DIR)/karma \ $(MAGIC_FRAGMENT_DIR)/kde \ $(MAGIC_FRAGMENT_DIR)/keepass \ $(MAGIC_FRAGMENT_DIR)/kerberos \ +$(MAGIC_FRAGMENT_DIR)/kicad \ $(MAGIC_FRAGMENT_DIR)/kml \ $(MAGIC_FRAGMENT_DIR)/lecter \ $(MAGIC_FRAGMENT_DIR)/lex \ @@ -419,8 +426,8 @@ $(MAGIC_FRAGMENT_DIR)/lua \ $(MAGIC_FRAGMENT_DIR)/luks \ $(MAGIC_FRAGMENT_DIR)/m4 \ $(MAGIC_FRAGMENT_DIR)/mach \ -$(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/macintosh \ +$(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/magic \ $(MAGIC_FRAGMENT_DIR)/mail.news \ $(MAGIC_FRAGMENT_DIR)/make \ @@ -448,8 +455,8 @@ $(MAGIC_FRAGMENT_DIR)/motorola \ $(MAGIC_FRAGMENT_DIR)/mozilla \ $(MAGIC_FRAGMENT_DIR)/msdos \ $(MAGIC_FRAGMENT_DIR)/msooxml \ -$(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/msvc \ +$(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/mup \ $(MAGIC_FRAGMENT_DIR)/music \ $(MAGIC_FRAGMENT_DIR)/nasa \ @@ -461,6 +468,7 @@ $(MAGIC_FRAGMENT_DIR)/netscape \ $(MAGIC_FRAGMENT_DIR)/netware \ $(MAGIC_FRAGMENT_DIR)/news \ $(MAGIC_FRAGMENT_DIR)/nitpicker \ +$(MAGIC_FRAGMENT_DIR)/numpy \ $(MAGIC_FRAGMENT_DIR)/oasis \ $(MAGIC_FRAGMENT_DIR)/ocaml \ $(MAGIC_FRAGMENT_DIR)/octave \ @@ -500,6 +508,7 @@ $(MAGIC_FRAGMENT_DIR)/revision \ $(MAGIC_FRAGMENT_DIR)/riff \ $(MAGIC_FRAGMENT_DIR)/rpi \ $(MAGIC_FRAGMENT_DIR)/rpm \ +$(MAGIC_FRAGMENT_DIR)/rpmsg \ $(MAGIC_FRAGMENT_DIR)/rtf \ $(MAGIC_FRAGMENT_DIR)/ruby \ $(MAGIC_FRAGMENT_DIR)/sc \ @@ -543,8 +552,8 @@ $(MAGIC_FRAGMENT_DIR)/unicode \ $(MAGIC_FRAGMENT_DIR)/unknown \ $(MAGIC_FRAGMENT_DIR)/uterus \ $(MAGIC_FRAGMENT_DIR)/uuencode \ -$(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/vacuum-cleaner \ +$(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/varied.script \ $(MAGIC_FRAGMENT_DIR)/vax \ $(MAGIC_FRAGMENT_DIR)/vicar \ @@ -572,7 +581,7 @@ $(MAGIC_FRAGMENT_DIR)/yara \ $(MAGIC_FRAGMENT_DIR)/zfs \ $(MAGIC_FRAGMENT_DIR)/zilog \ $(MAGIC_FRAGMENT_DIR)/zip \ -$(MAGIC_FRAGMENT_DIR)/zyxel +$(MAGIC_FRAGMENT_DIR)/zyxel MAGIC = magic.mgc CLEANFILES = ${MAGIC} $(MAGIC_FRAGMENT_DIR)/Localstuff diff --git a/src/Makefile.am b/src/Makefile.am index bbd3f5016d5c..2fbefdbfcb5b 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -8,14 +8,14 @@ AM_CPPFLAGS = -DMAGIC='"$(MAGIC)"' AM_CFLAGS = $(CFLAG_VISIBILITY) @WARNINGS@ libmagic_la_SOURCES = buffer.c magic.c apprentice.c softmagic.c ascmagic.c \ - encoding.c compress.c is_tar.c readelf.c print.c fsmagic.c \ + encoding.c compress.c is_json.c is_tar.c readelf.c print.c fsmagic.c \ funcs.c file.h readelf.h tar.h apptype.c der.c der.h \ file_opts.h elfclass.h mygetopt.h cdf.c cdf_time.c readcdf.c cdf.h libmagic_la_LDFLAGS = -no-undefined -version-info 1:0:0 if MINGW MINGWLIBS = -lgnurx -lshlwapi else -MINGWLIBS = +MINGWLIBS = endif libmagic_la_LIBADD = $(LTLIBOBJS) $(MINGWLIBS) diff --git a/src/Makefile.in b/src/Makefile.in index afa7bfa061eb..29567c470280 100644 --- a/src/Makefile.in +++ b/src/Makefile.in @@ -136,9 +136,9 @@ LTLIBRARIES = $(lib_LTLIBRARIES) am__DEPENDENCIES_1 = libmagic_la_DEPENDENCIES = $(LTLIBOBJS) $(am__DEPENDENCIES_1) am_libmagic_la_OBJECTS = buffer.lo magic.lo apprentice.lo softmagic.lo \ - ascmagic.lo encoding.lo compress.lo is_tar.lo readelf.lo \ - print.lo fsmagic.lo funcs.lo apptype.lo der.lo cdf.lo \ - cdf_time.lo readcdf.lo + ascmagic.lo encoding.lo compress.lo is_json.lo is_tar.lo \ + readelf.lo print.lo fsmagic.lo funcs.lo apptype.lo der.lo \ + cdf.lo cdf_time.lo readcdf.lo libmagic_la_OBJECTS = $(am_libmagic_la_OBJECTS) AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) @@ -343,7 +343,7 @@ nodist_include_HEADERS = magic.h AM_CPPFLAGS = -DMAGIC='"$(MAGIC)"' AM_CFLAGS = $(CFLAG_VISIBILITY) @WARNINGS@ libmagic_la_SOURCES = buffer.c magic.c apprentice.c softmagic.c ascmagic.c \ - encoding.c compress.c is_tar.c readelf.c print.c fsmagic.c \ + encoding.c compress.c is_json.c is_tar.c readelf.c print.c fsmagic.c \ funcs.c file.h readelf.h tar.h apptype.c der.c der.h \ file_opts.h elfclass.h mygetopt.h cdf.c cdf_time.c readcdf.c cdf.h @@ -515,6 +515,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/file.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/fsmagic.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/funcs.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/is_json.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/is_tar.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/magic.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/print.Plo@am__quote@ diff --git a/src/apprentice.c b/src/apprentice.c index ea8d0b222e0a..eca3ae06af99 100644 --- a/src/apprentice.c +++ b/src/apprentice.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: apprentice.c,v 1.272 2018/06/22 20:39:50 christos Exp $") +FILE_RCSID("@(#)$File: apprentice.c,v 1.283 2019/02/20 02:35:27 christos Exp $") #endif /* lint */ #include "magic.h" @@ -40,9 +40,7 @@ FILE_RCSID("@(#)$File: apprentice.c,v 1.272 2018/06/22 20:39:50 christos Exp $") #ifdef HAVE_UNISTD_H #include <unistd.h> #endif -#ifdef HAVE_STDDEF_H #include <stddef.h> -#endif #include <string.h> #include <assert.h> #include <ctype.h> @@ -51,20 +49,13 @@ FILE_RCSID("@(#)$File: apprentice.c,v 1.272 2018/06/22 20:39:50 christos Exp $") #include <sys/mman.h> #endif #include <dirent.h> -#if defined(HAVE_LIMITS_H) #include <limits.h> -#endif -#ifndef SSIZE_MAX -#define MAXMAGIC_SIZE ((ssize_t)0x7fffffff) -#else -#define MAXMAGIC_SIZE SSIZE_MAX -#endif -#define EATAB {while (isascii((unsigned char) *l) && \ - isspace((unsigned char) *l)) ++l;} -#define LOWCASE(l) (isupper((unsigned char) (l)) ? \ - tolower((unsigned char) (l)) : (l)) +#define EATAB {while (isascii(CAST(unsigned char, *l)) && \ + isspace(CAST(unsigned char, *l))) ++l;} +#define LOWCASE(l) (isupper(CAST(unsigned char, l)) ? \ + tolower(CAST(unsigned char, l)) : (l)) /* * Work around a bug in headers on Digital Unix. * At least confirmed for: OSF1 V4.0 878 @@ -83,15 +74,15 @@ FILE_RCSID("@(#)$File: apprentice.c,v 1.272 2018/06/22 20:39:50 christos Exp $") #define MAP_FILE 0 #endif -#define ALLOC_CHUNK (size_t)10 -#define ALLOC_INCR (size_t)200 +#define ALLOC_CHUNK CAST(size_t, 10) +#define ALLOC_INCR CAST(size_t, 200) #define MAP_TYPE_USER 0 #define MAP_TYPE_MALLOC 1 #define MAP_TYPE_MMAP 2 struct magic_entry { - struct magic *mp; + struct magic *mp; uint32_t cont_count; uint32_t max_count; }; @@ -126,7 +117,7 @@ private int apprentice_1(struct magic_set *, const char *, int); private size_t apprentice_magic_strength(const struct magic *); private int apprentice_sort(const void *, const void *); private void apprentice_list(struct mlist *, int ); -private struct magic_map *apprentice_load(struct magic_set *, +private struct magic_map *apprentice_load(struct magic_set *, const char *, int); private struct mlist *mlist_alloc(void); private void mlist_free(struct mlist *); @@ -300,12 +291,21 @@ get_type(const struct type_tbl_s *tbl, const char *l, const char **t) return p->type; } +private off_t +maxoff_t(void) { + if (/*CONSTCOND*/sizeof(off_t) == sizeof(int)) + return CAST(off_t, INT_MAX); + if (/*CONSTCOND*/sizeof(off_t) == sizeof(long)) + return CAST(off_t, LONG_MAX); + return 0x7fffffff; +} + private int get_standard_integer_type(const char *l, const char **t) { int type; - if (isalpha((unsigned char)l[1])) { + if (isalpha(CAST(unsigned char, l[1]))) { switch (l[1]) { case 'C': /* "dC" and "uC" */ @@ -340,7 +340,7 @@ get_standard_integer_type(const char *l, const char **t) return FILE_INVALID; } l += 2; - } else if (isdigit((unsigned char)l[1])) { + } else if (isdigit(CAST(unsigned char, l[1]))) { /* * "d{num}" and "u{num}"; we only support {num} values * of 1, 2, 4, and 8 - the Single UNIX Specification @@ -351,7 +351,7 @@ get_standard_integer_type(const char *l, const char **t) * neither of them support values bigger than 8 or * non-power-of-2 values. */ - if (isdigit((unsigned char)l[2])) { + if (isdigit(CAST(unsigned char, l[2]))) { /* Multi-digit, so > 9 */ return FILE_INVALID; } @@ -437,8 +437,8 @@ apprentice_1(struct magic_set *ms, const char *fn, int action) if (magicsize != FILE_MAGICSIZE) { file_error(ms, 0, "magic element size %lu != %lu", - (unsigned long)sizeof(*map->magic[0]), - (unsigned long)FILE_MAGICSIZE); + CAST(unsigned long, sizeof(*map->magic[0])), + CAST(unsigned long, FILE_MAGICSIZE)); return -1; } @@ -451,7 +451,7 @@ apprentice_1(struct magic_set *ms, const char *fn, int action) #ifndef COMPILE_ONLY map = apprentice_map(ms, fn); - if (map == (struct magic_map *)-1) + if (map == RCAST(struct magic_map *, -1)) return -1; if (map == NULL) { if (ms->flags & MAGIC_CHECK) @@ -503,7 +503,7 @@ file_ms_alloc(int flags) struct magic_set *ms; size_t i, len; - if ((ms = CAST(struct magic_set *, calloc((size_t)1, + if ((ms = CAST(struct magic_set *, calloc(CAST(size_t, 1u), sizeof(struct magic_set)))) == NULL) return NULL; @@ -582,6 +582,14 @@ mlist_alloc(void) } private void +mlist_free_one(struct mlist *ml) +{ + if (ml->map) + apprentice_unmap(CAST(struct magic_map *, ml->map)); + free(ml); +} + +private void mlist_free(struct mlist *mlist) { struct mlist *ml, *next; @@ -589,14 +597,11 @@ mlist_free(struct mlist *mlist) if (mlist == NULL) return; - ml = mlist->next; - for (ml = mlist->next; (next = ml->next) != NULL; ml = next) { - if (ml->map) - apprentice_unmap(CAST(struct magic_map *, ml->map)); - free(ml); - if (ml == mlist) - break; + for (ml = mlist->next; ml != mlist; ml = next) { + next = ml->next; + mlist_free_one(ml); } + mlist_free_one(mlist); } #ifndef COMPILE_ONLY @@ -826,7 +831,7 @@ typesize(int type) case FILE_LEDOUBLE: return 8; default: - return (size_t)~0; + return CAST(size_t, ~0); } } @@ -836,8 +841,9 @@ typesize(int type) private size_t apprentice_magic_strength(const struct magic *m) { -#define MULT 10 - size_t ts, v, val = 2 * MULT; /* baseline strength */ +#define MULT 10U + size_t ts, v; + ssize_t val = 2 * MULT; /* baseline strength */ switch (m->type) { case FILE_DEFAULT: /* make sure this sorts last */ @@ -880,7 +886,7 @@ apprentice_magic_strength(const struct magic *m) case FILE_BEDOUBLE: case FILE_LEDOUBLE: ts = typesize(m->type); - if (ts == (size_t)~0) + if (ts == CAST(size_t, ~0)) abort(); val += ts * MULT; break; @@ -896,6 +902,8 @@ apprentice_magic_strength(const struct magic *m) break; case FILE_SEARCH: + if (m->vallen == 0) + break; val += m->vallen * MAX(MULT / m->vallen, 1); break; @@ -943,9 +951,6 @@ apprentice_magic_strength(const struct magic *m) abort(); } - if (val == 0) /* ensure we only return 0 for FILE_DEFAULT */ - val = 1; - switch (m->factor_op) { case FILE_FACTOR_OP_NONE: break; @@ -965,6 +970,9 @@ apprentice_magic_strength(const struct magic *m) abort(); } + if (val <= 0) /* ensure we only return 0 for FILE_DEFAULT */ + val = 1; + /* * Magic entries with no description get a bonus because they depend * on subsequent magic entries to print something. @@ -974,7 +982,7 @@ apprentice_magic_strength(const struct magic *m) return val; } -/* +/* * Sort callback for sorting entries by "strength" (basically length) */ private int @@ -992,7 +1000,7 @@ apprentice_sort(const void *a, const void *b) return 1; } -/* +/* * Shows sorted patterns list in the order which is used for the matching */ private void @@ -1088,12 +1096,12 @@ set_test_type(struct magic *mstart, struct magic *m) mstart->flag |= BINTEST; if (mstart->str_flags & STRING_TEXTTEST) mstart->flag |= TEXTTEST; - + if (mstart->flag & (TEXTTEST|BINTEST)) break; /* binary test if pattern is not text */ - if (file_looks_utf8(m->value.us, (size_t)m->vallen, NULL, + if (file_looks_utf8(m->value.us, CAST(size_t, m->vallen), NULL, NULL) <= 0) mstart->flag |= BINTEST; else @@ -1174,7 +1182,7 @@ load_1(struct magic_set *ms, int action, const char *fn, int *errs, size_t i; for (i = 0; bang[i].name != NULL; i++) { - if ((size_t)(len - 2) > bang[i].len && + if (CAST(size_t, len - 2) > bang[i].len && memcmp(bang[i].name, line + 2, bang[i].len) == 0) break; @@ -1227,7 +1235,7 @@ load_1(struct magic_set *ms, int action, const char *fn, int *errs, private int cmpstrp(const void *p1, const void *p2) { - return strcmp(*(char *const *)p1, *(char *const *)p2); + return strcmp(*RCAST(char *const *, p1), *RCAST(char *const *, p2)); } @@ -1253,10 +1261,10 @@ set_text_binary(struct magic_set *ms, struct magic_entry *me, uint32_t nme, if (me[i].mp->flag & BINTEST) { char *p = strstr(me[i].mp->desc, text); if (p && (p == me[i].mp->desc || - isspace((unsigned char)p[-1])) && + isspace(CAST(unsigned char, p[-1]))) && (p + len - me[i].mp->desc == MAXstring || (p[len] == '\0' || - isspace((unsigned char)p[len])))) + isspace(CAST(unsigned char, p[len]))))) (void)fprintf(stderr, "*** Possible " "binary test for text type\n"); } @@ -1280,7 +1288,7 @@ set_last_default(struct magic_set *ms, struct magic_entry *me, uint32_t nme) file_magwarn(ms, "level 0 \"default\" did not sort last"); } - return; + return; } } } @@ -1387,12 +1395,14 @@ apprentice_load(struct magic_set *ms, const char *fn, int action) filearr[files++] = mfn; } closedir(dir); - qsort(filearr, files, sizeof(*filearr), cmpstrp); - for (i = 0; i < files; i++) { - load_1(ms, action, filearr[i], &errs, mset); - free(filearr[i]); + if (filearr) { + qsort(filearr, files, sizeof(*filearr), cmpstrp); + for (i = 0; i < files; i++) { + load_1(ms, action, filearr[i], &errs, mset); + free(filearr[i]); + } + free(filearr); } - free(filearr); } else load_1(ms, action, fn, &errs, mset); if (errs) @@ -1450,12 +1460,12 @@ file_signextend(struct magic_set *ms, struct magic *m, uint64_t v) * the sign extension must have happened. */ case FILE_BYTE: - v = (signed char) v; + v = CAST(signed char, v); break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: - v = (short) v; + v = CAST(short, v); break; case FILE_DATE: case FILE_BEDATE: @@ -1472,7 +1482,7 @@ file_signextend(struct magic_set *ms, struct magic *m, uint64_t v) case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: - v = (int32_t) v; + v = CAST(int32_t, v); break; case FILE_QUAD: case FILE_BEQUAD: @@ -1489,7 +1499,7 @@ file_signextend(struct magic_set *ms, struct magic *m, uint64_t v) case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: - v = (int64_t) v; + v = CAST(int64_t, v); break; case FILE_STRING: case FILE_PSTRING: @@ -1616,7 +1626,7 @@ get_cond(const char *l, const char **t) for (p = cond_tbl; p->len; p++) { if (strncmp(l, p->name, p->len) == 0 && - isspace((unsigned char)l[p->len])) { + isspace(CAST(unsigned char, l[p->len]))) { if (t) *t = l + p->len; break; @@ -1674,7 +1684,7 @@ parse_indirect_modifier(struct magic_set *ms, struct magic *m, const char **lp) { const char *l = *lp; - while (!isspace((unsigned char)*++l)) + while (!isspace(CAST(unsigned char, *++l))) switch (*l) { case CHAR_INDIRECT_RELATIVE: m->str_flags |= INDIRECT_RELATIVE; @@ -1700,7 +1710,7 @@ parse_op_modifier(struct magic_set *ms, struct magic *m, const char **lp, ++l; m->mask_op |= op; - val = (uint64_t)strtoull(l, &t, 0); + val = CAST(uint64_t, strtoull(l, &t, 0)); l = t; m->num_mask = file_signextend(ms, m, val); eatsize(&l); @@ -1714,7 +1724,7 @@ parse_string_modifier(struct magic_set *ms, struct magic *m, const char **lp) char *t; int have_range = 0; - while (!isspace((unsigned char)*++l)) { + while (!isspace(CAST(unsigned char, *++l))) { switch (*l) { case '0': case '1': case '2': case '3': case '4': case '5': @@ -1796,7 +1806,7 @@ parse_string_modifier(struct magic_set *ms, struct magic *m, const char **lp) goto out; } /* allow multiple '/' for readability */ - if (l[1] == '/' && !isspace((unsigned char)l[2])) + if (l[1] == '/' && !isspace(CAST(unsigned char, l[2]))) l++; } if (string_modifier_check(ms, m) == -1) @@ -1833,7 +1843,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, */ while (*l == '>') { ++l; /* step over */ - cont_level++; + cont_level++; } #ifdef ENABLE_CONDITIONALS if (cont_level == 0 || cont_level > last_cont_level) @@ -1851,7 +1861,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, return -1; } m = &me->mp[me->cont_count - 1]; - diff = (int32_t)cont_level - (int32_t)m->cont_level; + diff = CAST(int32_t, cont_level) - CAST(int32_t, m->cont_level); if (diff > 1) file_magwarn(ms, "New continuation level %u is more " "than one larger than current level %u", cont_level, @@ -1864,7 +1874,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, file_oomem(ms, sizeof(*nm) * cnt); return -1; } - me->mp = m = nm; + me->mp = nm; me->max_count = CAST(uint32_t, cnt); } m = &me->mp[me->cont_count++]; @@ -1910,7 +1920,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, } /* get offset, then skip over it */ - m->offset = (int32_t)strtol(l, &t, 0); + m->offset = CAST(int32_t, strtol(l, &t, 0)); if (l == t) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "offset `%s' invalid", l); @@ -2008,8 +2018,8 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, m->in_op |= FILE_OPINDIRECT; l++; } - if (isdigit((unsigned char)*l) || *l == '-') { - m->in_offset = (int32_t)strtol(l, &t, 0); + if (isdigit(CAST(unsigned char, *l)) || *l == '-') { + m->in_offset = CAST(int32_t, strtol(l, &t, 0)); if (l == t) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, @@ -2018,7 +2028,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, } l = t; } - if (*l++ != ')' || + if (*l++ != ')' || ((m->in_op & FILE_OPINDIRECT) && *l++ != ')')) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, @@ -2043,7 +2053,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, /* * Try it as a keyword type prefixed by "u"; match what * follows the "u". If that fails, try it as an SUS - * integer type. + * integer type. */ m->type = get_type(type_tbl, l + 1, &l); if (m->type == FILE_INVALID) { @@ -2072,7 +2082,8 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, */ if (*l == 'd') m->type = get_standard_integer_type(l, &l); - else if (*l == 's' && !isalpha((unsigned char)l[1])) { + else if (*l == 's' + && !isalpha(CAST(unsigned char, l[1]))) { m->type = FILE_STRING; ++l; } @@ -2083,7 +2094,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, /* Not found - try it as a special keyword. */ m->type = get_type(special_tbl, l, &l); } - + if (m->type == FILE_INVALID) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "type `%s' invalid", l); @@ -2130,7 +2141,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, * anything if mask = 0 (unless you have a better idea) */ EATAB; - + switch (*l) { case '>': case '<': @@ -2162,8 +2173,8 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, break; default: m->reln = '='; /* the default relation */ - if (*l == 'x' && ((isascii((unsigned char)l[1]) && - isspace((unsigned char)l[1])) || !l[1])) { + if (*l == 'x' && ((isascii(CAST(unsigned char, l[1])) && + isspace(CAST(unsigned char, l[1]))) || !l[1])) { m->reln = *l; ++l; } @@ -2177,7 +2188,7 @@ parse(struct magic_set *ms, struct magic_entry *me, const char *line, /* * TODO finish this macro and start using it! - * #define offsetcheck {if (offset > ms->bytes_max -1) + * #define offsetcheck {if (offset > ms->bytes_max -1) * magwarn("offset too big"); } */ @@ -2260,11 +2271,11 @@ parse_strength(struct magic_set *ms, struct magic_entry *me, const char *line) file_magwarn(ms, "Too large factor `%lu'", factor); goto out; } - if (*el && !isspace((unsigned char)*el)) { + if (*el && !isspace(CAST(unsigned char, *el))) { file_magwarn(ms, "Bad factor `%s'", l); goto out; } - m->factor = (uint8_t)factor; + m->factor = CAST(uint8_t, factor); if (m->factor == 0 && m->factor_op == FILE_FACTOR_OP_DIV) { file_magwarn(ms, "Cannot have factor op `%c' and factor %u", m->factor_op, m->factor); @@ -2295,9 +2306,9 @@ parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line, if (buf[0] != '\0') { len = nt ? strlen(buf) : len; file_magwarn(ms, "Current entry already has a %s type " - "`%.*s', new type `%s'", name, (int)len, buf, l); + "`%.*s', new type `%s'", name, CAST(int, len), buf, l); return -1; - } + } if (*m->desc == '\0') { file_magwarn(ms, "Current entry does not yet have a " @@ -2316,7 +2327,7 @@ parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line, file_magwarn(ms, "%s type `%s' truncated %" SIZE_T_FORMAT "u", name, line, i); } else { - if (!isspace((unsigned char)*l) && !goodchar(*l, extra)) + if (!isspace(CAST(unsigned char, *l)) && !goodchar(*l, extra)) file_magwarn(ms, "%s type `%s' has bad char '%c'", name, line, *l); if (nt) @@ -2418,7 +2429,7 @@ check_format_type(const char *ptr, int type, const char **estr) if (*ptr == '#') ptr++; #define CHECKLEN() do { \ - for (len = cnt = 0; isdigit((unsigned char)*ptr); ptr++, cnt++) \ + for (len = cnt = 0; isdigit(CAST(unsigned char, *ptr)); ptr++, cnt++) \ len = len * 10 + (*ptr - '0'); \ if (cnt > 5 || len > 1024) \ goto toolong; \ @@ -2434,7 +2445,7 @@ check_format_type(const char *ptr, int type, const char **estr) if (*ptr++ != 'l') goto invalid; } - + switch (*ptr++) { #ifdef STRICT_FORMAT /* "long" formats are int formats for us */ /* so don't accept the 'l' modifier */ @@ -2452,7 +2463,7 @@ check_format_type(const char *ptr, int type, const char **estr) default: goto invalid; } - + /* * Don't accept h and hh modifiers. They make writing * magic entries more complicated, for very little benefit @@ -2508,7 +2519,7 @@ check_format_type(const char *ptr, int type, const char **estr) default: goto invalid; } - + case FILE_FMT_FLOAT: case FILE_FMT_DOUBLE: if (*ptr == '-') @@ -2527,30 +2538,30 @@ check_format_type(const char *ptr, int type, const char **estr) case 'g': case 'G': return 0; - + default: goto invalid; } - + case FILE_FMT_STR: if (*ptr == '-') ptr++; - while (isdigit((unsigned char )*ptr)) + while (isdigit(CAST(unsigned char, *ptr))) ptr++; if (*ptr == '.') { ptr++; - while (isdigit((unsigned char )*ptr)) + while (isdigit(CAST(unsigned char , *ptr))) ptr++; } - + switch (*ptr++) { case 's': return 0; default: goto invalid; } - + default: /* internal error */ abort(); @@ -2561,7 +2572,7 @@ toolong: *estr = "too long"; return -1; } - + /* * Check that the optional printf format in description matches * the type of the magic. @@ -2584,7 +2595,7 @@ check_format(struct magic_set *ms, struct magic *m) if (m->type >= file_nformats) { file_magwarn(ms, "Internal error inconsistency between " - "m->type and format strings"); + "m->type and format strings"); return -1; } if (file_formats[m->type] == FILE_FMT_NONE) { @@ -2604,7 +2615,7 @@ check_format(struct magic_set *ms, struct magic *m) file_names[m->type], m->desc); return -1; } - + for (; *ptr; ptr++) { if (*ptr == '%') { file_magwarn(ms, @@ -2617,9 +2628,9 @@ check_format(struct magic_set *ms, struct magic *m) return 0; } -/* - * Read a numeric value from a pointer, into the value union of a magic - * pointer, according to the magic type. Update the string pointer to point +/* + * Read a numeric value from a pointer, into the value union of a magic + * pointer, according to the magic type. Update the string pointer to point * just after the number read. Return 0 for success, non-zero for failure. */ private int @@ -2685,7 +2696,7 @@ getvalue(struct magic_set *ms, struct magic *m, const char **p, int action) return 0; default: errno = 0; - ull = (uint64_t)strtoull(*p, &ep, 0); + ull = CAST(uint64_t, strtoull(*p, &ep, 0)); m->value.q = file_signextend(ms, m, ull); if (*p == ep) { file_magwarn(ms, "Unparseable number `%s'", *p); @@ -2694,24 +2705,24 @@ getvalue(struct magic_set *ms, struct magic *m, const char **p, int action) uint64_t x; const char *q; - if (ts == (size_t)~0) { + if (ts == CAST(size_t, ~0)) { file_magwarn(ms, "Expected numeric type got `%s'", type_tbl[m->type].name); } - for (q = *p; isspace((unsigned char)*q); q++) + for (q = *p; isspace(CAST(unsigned char, *q)); q++) continue; if (*q == '-') - ull = -(int64_t)ull; + ull = -CAST(int64_t, ull); switch (ts) { case 1: - x = (uint64_t)(ull & ~0xffULL); + x = CAST(uint64_t, ull & ~0xffULL); break; case 2: - x = (uint64_t)(ull & ~0xffffULL); + x = CAST(uint64_t, ull & ~0xffffULL); break; case 4: - x = (uint64_t)(ull & ~0xffffffffULL); + x = CAST(uint64_t, ull & ~0xffffffffULL); break; case 8: x = 0; @@ -2751,7 +2762,7 @@ getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) int val; while ((c = *s++) != '\0') { - if (isspace((unsigned char) c)) + if (isspace(CAST(unsigned char, c))) break; if (p >= pmax) { file_error(ms, 0, "string too long: `%s'", origs); @@ -2775,8 +2786,8 @@ getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) /*FALLTHROUGH*/ default: if (warn) { - if (isprint((unsigned char)c)) { - /* Allow escaping of + if (isprint(CAST(unsigned char, c))) { + /* Allow escaping of * ``relations'' */ if (strchr("<>&^=!", c) == NULL && (m->type != FILE_REGEX || @@ -2813,7 +2824,7 @@ getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) case '!': /* and baskslash itself */ case '\\': - *p++ = (char) c; + *p++ = CAST(char, c); break; case 'a': @@ -2865,7 +2876,7 @@ getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) } else --s; - *p++ = (char)val; + *p++ = CAST(char, val); break; /* \x and up to 2 hex digits */ @@ -2881,18 +2892,18 @@ getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) --s; } else --s; - *p++ = (char)val; + *p++ = CAST(char, val); break; } } else - *p++ = (char)c; + *p++ = CAST(char, c); } --s; out: *p = '\0'; m->vallen = CAST(unsigned char, (p - origp)); if (m->type == FILE_PSTRING) - m->vallen += (unsigned char)file_pstring_length_size(m); + m->vallen += CAST(unsigned char, file_pstring_length_size(m)); return s; } @@ -2901,9 +2912,9 @@ out: private int hextoint(int c) { - if (!isascii((unsigned char) c)) + if (!isascii(CAST(unsigned char, c))) return -1; - if (isdigit((unsigned char) c)) + if (isdigit(CAST(unsigned char, c))) return c - '0'; if ((c >= 'a') && (c <= 'f')) return c + 10 - 'a'; @@ -2981,7 +2992,7 @@ eatsize(const char **p) { const char *l = *p; - if (LOWCASE(*l) == 'u') + if (LOWCASE(*l) == 'u') l++; switch (LOWCASE(*l)) { @@ -3052,17 +3063,17 @@ apprentice_map(struct magic_set *ms, const char *fn) file_error(ms, errno, "cannot stat `%s'", dbname); goto error; } - if (st.st_size < 8 || st.st_size > MAXMAGIC_SIZE) { + if (st.st_size < 8 || st.st_size > maxoff_t()) { file_error(ms, 0, "file `%s' is too %s", dbname, st.st_size < 8 ? "small" : "large"); goto error; } - map->len = (size_t)st.st_size; + map->len = CAST(size_t, st.st_size); #ifdef QUICK map->type = MAP_TYPE_MMAP; - if ((map->p = mmap(0, (size_t)st.st_size, PROT_READ|PROT_WRITE, - MAP_PRIVATE|MAP_FILE, fd, (off_t)0)) == MAP_FAILED) { + if ((map->p = mmap(0, CAST(size_t, st.st_size), PROT_READ|PROT_WRITE, + MAP_PRIVATE|MAP_FILE, fd, CAST(off_t, 0))) == MAP_FAILED) { file_error(ms, errno, "cannot map `%s'", dbname); goto error; } @@ -3082,11 +3093,11 @@ apprentice_map(struct magic_set *ms, const char *fn) fd = -1; if (check_buffer(ms, map, dbname) != 0) { - rv = (struct magic_map *)-1; + rv = RCAST(struct magic_map *, -1); goto error; } #ifdef QUICK - if (mprotect(map->p, (size_t)st.st_size, PROT_READ) == -1) { + if (mprotect(map->p, CAST(size_t, st.st_size), PROT_READ) == -1) { file_error(ms, errno, "cannot mprotect `%s'", dbname); goto error; } @@ -3130,7 +3141,7 @@ check_buffer(struct magic_set *ms, struct magic_map *map, const char *dbname) VERSIONNO, dbname, version); return -1; } - entries = (uint32_t)(map->len / sizeof(struct magic)); + entries = CAST(uint32_t, map->len / sizeof(struct magic)); if ((entries * sizeof(struct magic)) != map->len) { file_error(ms, 0, "Size of `%s' %" SIZE_T_FORMAT "u is not " "a multiple of %" SIZE_T_FORMAT "u", @@ -3179,10 +3190,10 @@ apprentice_compile(struct magic_set *ms, struct magic_map *map, const char *fn) dbname = mkdbname(ms, fn, 1); - if (dbname == NULL) + if (dbname == NULL) goto out; - if ((fd = open(dbname, O_WRONLY|O_CREAT|O_TRUNC|O_BINARY, 0644)) == -1) + if ((fd = open(dbname, O_WRONLY|O_CREAT|O_TRUNC|O_BINARY, 0644)) == -1) { file_error(ms, errno, "cannot open `%s'", dbname); goto out; @@ -3192,14 +3203,14 @@ apprentice_compile(struct magic_set *ms, struct magic_map *map, const char *fn) hdr.h[1] = VERSIONNO; memcpy(hdr.h + 2, map->nmagic, nm); - if (write(fd, &hdr, sizeof(hdr)) != (ssize_t)sizeof(hdr)) { + if (write(fd, &hdr, sizeof(hdr)) != CAST(ssize_t, sizeof(hdr))) { file_error(ms, errno, "error writing `%s'", dbname); goto out2; } for (i = 0; i < MAGIC_SETS; i++) { len = m * map->nmagic[i]; - if (write(fd, map->magic[i], len) != (ssize_t)len) { + if (write(fd, map->magic[i], len) != CAST(ssize_t, len)) { file_error(ms, errno, "error writing `%s'", dbname); goto out2; } @@ -3245,7 +3256,8 @@ mkdbname(struct magic_set *ms, const char *fn, int strip) q++; /* Compatibility with old code that looked in .mime */ if (ms->flags & MAGIC_MIME) { - if (asprintf(&buf, "%.*s.mime%s", (int)(q - fn), fn, ext) < 0) + if (asprintf(&buf, "%.*s.mime%s", CAST(int, q - fn), fn, ext) + < 0) return NULL; if (access(buf, R_OK) != -1) { ms->flags &= MAGIC_MIME_TYPE; @@ -3253,7 +3265,7 @@ mkdbname(struct magic_set *ms, const char *fn, int strip) } free(buf); } - if (asprintf(&buf, "%.*s%s", (int)(q - fn), fn, ext) < 0) + if (asprintf(&buf, "%.*s%s", CAST(int, q - fn), fn, ext) < 0) return NULL; /* Compatibility with old code that looked in .mime */ @@ -3280,8 +3292,8 @@ private uint16_t swap2(uint16_t sv) { uint16_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); d[0] = s[1]; d[1] = s[0]; return rv; @@ -3294,8 +3306,8 @@ private uint32_t swap4(uint32_t sv) { uint32_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); d[0] = s[3]; d[1] = s[2]; d[2] = s[1]; @@ -3310,8 +3322,8 @@ private uint64_t swap8(uint64_t sv) { uint64_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); #if 0 d[0] = s[3]; d[1] = s[2]; @@ -3341,9 +3353,9 @@ private void bs1(struct magic *m) { m->cont_level = swap2(m->cont_level); - m->offset = swap4((uint32_t)m->offset); - m->in_offset = swap4((uint32_t)m->in_offset); - m->lineno = swap4((uint32_t)m->lineno); + m->offset = swap4(CAST(uint32_t, m->offset)); + m->in_offset = swap4(CAST(uint32_t, m->in_offset)); + m->lineno = swap4(CAST(uint32_t, m->lineno)); if (IS_STRING(m->type)) { m->str_range = swap4(m->str_range); m->str_flags = swap4(m->str_flags); @@ -3354,7 +3366,7 @@ bs1(struct magic *m) } } -protected size_t +protected size_t file_pstring_length_size(const struct magic *m) { switch (m->str_flags & PSTRING_LEN) { @@ -3375,7 +3387,7 @@ protected size_t file_pstring_get_length(const struct magic *m, const char *ss) { size_t len = 0; - const unsigned char *s = (const unsigned char *)ss; + const unsigned char *s = RCAST(const unsigned char *, ss); unsigned int s3, s2, s1, s0; switch (m->str_flags & PSTRING_LEN) { diff --git a/src/apptype.c b/src/apptype.c index fcd764e49a44..1bb33e41c27a 100644 --- a/src/apptype.c +++ b/src/apptype.c @@ -1,15 +1,15 @@ /* * Adapted from: apptype.c, Written by Eberhard Mattes and put into the * public domain - * + * * Notes: 1. Qualify the filename so that DosQueryAppType does not do extraneous * searches. - * + * * 2. DosQueryAppType will return FAPPTYP_DOS on a file ending with ".com" * (other than an OS/2 exe or Win exe with this name). Eberhard Mattes * remarks Tue, 6 Apr 93: Moreover, it reports the type of the (new and very * bug ridden) Win Emacs as "OS/2 executable". - * + * * 3. apptype() uses the filename if given, otherwise a tmp file is created with * the contents of buf. If buf is not the complete file, apptype can * incorrectly identify the exe type. The "-z" option of "file" is the reason @@ -18,16 +18,16 @@ /* * amai: Darrel Hankerson did the changes described here. - * + * * It remains to check the validity of comments (2.) since it's referred to an * "old" OS/2 version. - * + * */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: apptype.c,v 1.13 2011/09/07 21:57:15 christos Exp $") +FILE_RCSID("@(#)$File: apptype.c,v 1.14 2018/09/09 20:33:28 christos Exp $") #endif /* lint */ #include <stdlib.h> diff --git a/src/ascmagic.c b/src/ascmagic.c index 2d1abe543e60..624ac90b1f1c 100644 --- a/src/ascmagic.c +++ b/src/ascmagic.c @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: ascmagic.c,v 1.98 2017/11/02 20:25:39 christos Exp $") +FILE_RCSID("@(#)$File: ascmagic.c,v 1.104 2019/05/07 02:27:11 christos Exp $") #endif /* lint */ #include "magic.h" @@ -80,7 +80,13 @@ file_ascmagic(struct magic_set *ms, const struct buffer *b, int text) const char *type = NULL; bb = *b; - bb.flen = trim_nuls(b->fbuf, b->flen); + bb.flen = trim_nuls(CAST(const unsigned char *, b->fbuf), b->flen); + /* + * Avoid trimming at an odd byte if the original buffer was evenly + * sized; this avoids losing the last character on UTF-16 LE text + */ + if ((bb.flen & 1) && !(b->flen & 1)) + bb.flen++; /* If file doesn't look like any sort of text, give up. */ if (file_encoding(ms, &bb, &ubuf, &ulen, &code, &code_mime, @@ -96,17 +102,18 @@ file_ascmagic(struct magic_set *ms, const struct buffer *b, int text) } protected int -file_ascmagic_with_encoding(struct magic_set *ms, +file_ascmagic_with_encoding(struct magic_set *ms, const struct buffer *b, unichar *ubuf, size_t ulen, const char *code, const char *type, int text) { struct buffer bb; - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; unsigned char *utf8_buf = NULL, *utf8_end; - size_t mlen, i; + size_t mlen, i, len; int rv = -1; int mime = ms->flags & MAGIC_MIME; + int need_separator = 0; const char *subtype = NULL; const char *subtype_mime = NULL; @@ -121,7 +128,7 @@ file_ascmagic_with_encoding(struct magic_set *ms, int n_nel = 0; int executable = 0; - size_t last_line_end = (size_t)-1; + size_t last_line_end = CAST(size_t, -1); int has_long_lines = 0; nbytes = trim_nuls(buf, nbytes); @@ -144,12 +151,14 @@ file_ascmagic_with_encoding(struct magic_set *ms, if ((utf8_end = encode_utf8(utf8_buf, mlen, ubuf, ulen)) == NULL) goto done; - buffer_init(&bb, b->fd, utf8_buf, - (size_t)(utf8_end - utf8_buf)); + buffer_init(&bb, b->fd, &b->st, utf8_buf, + CAST(size_t, utf8_end - utf8_buf)); if ((rv = file_softmagic(ms, &bb, NULL, NULL, TEXTTEST, text)) == 0) rv = -1; + else + need_separator = 1; buffer_fini(&bb); if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION))) { rv = rv == -1 ? 0 : 1; @@ -200,8 +209,21 @@ file_ascmagic_with_encoding(struct magic_set *ms, rv = 0; goto done; } + len = file_printedlen(ms); if (mime) { - if (!file_printedlen(ms) && (mime & MAGIC_MIME_TYPE) != 0) { + if ((mime & MAGIC_MIME_TYPE) != 0) { + if (len) { + /* + * Softmagic printed something, we + * are either done, or we need a separator + */ + if ((ms->flags & MAGIC_CONTINUE) == 0) { + rv = 1; + goto done; + } + if (need_separator && file_separator(ms) == -1) + goto done; + } if (subtype_mime) { if (file_printf(ms, "%s", subtype_mime) == -1) goto done; @@ -211,7 +233,7 @@ file_ascmagic_with_encoding(struct magic_set *ms, } } } else { - if (file_printedlen(ms)) { + if (len) { switch (file_replace(ms, " text$", ", ")) { case 0: switch (file_replace(ms, " text executable$", @@ -324,42 +346,42 @@ encode_utf8(unsigned char *buf, size_t len, unichar *ubuf, size_t ulen) if (ubuf[i] <= 0x7f) { if (end - buf < 1) return NULL; - *buf++ = (unsigned char)ubuf[i]; + *buf++ = CAST(unsigned char, ubuf[i]); } else if (ubuf[i] <= 0x7ff) { if (end - buf < 2) return NULL; - *buf++ = (unsigned char)((ubuf[i] >> 6) + 0xc0); - *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] >> 6) + 0xc0); + *buf++ = CAST(unsigned char, (ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0xffff) { if (end - buf < 3) return NULL; - *buf++ = (unsigned char)((ubuf[i] >> 12) + 0xe0); - *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); - *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] >> 12) + 0xe0); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 6) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x1fffff) { if (end - buf < 4) return NULL; - *buf++ = (unsigned char)((ubuf[i] >> 18) + 0xf0); - *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); - *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] >> 18) + 0xf0); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 12) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 6) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x3ffffff) { if (end - buf < 5) return NULL; - *buf++ = (unsigned char)((ubuf[i] >> 24) + 0xf8); - *buf++ = (unsigned char)(((ubuf[i] >> 18) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); - *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] >> 24) + 0xf8); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 18) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 12) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 6) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x7fffffff) { if (end - buf < 6) return NULL; - *buf++ = (unsigned char)((ubuf[i] >> 30) + 0xfc); - *buf++ = (unsigned char)(((ubuf[i] >> 24) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 18) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); - *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); - *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] >> 30) + 0xfc); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 24) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 18) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 12) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, ((ubuf[i] >> 6) & 0x3f) + 0x80); + *buf++ = CAST(unsigned char, (ubuf[i] & 0x3f) + 0x80); } else /* Invalid character */ return NULL; } diff --git a/src/asprintf.c b/src/asprintf.c index 30852d2b3b00..2d14e8074f23 100644 --- a/src/asprintf.c +++ b/src/asprintf.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -29,7 +29,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: asprintf.c,v 1.4 2010/07/21 16:47:17 christos Exp $") +FILE_RCSID("@(#)$File: asprintf.c,v 1.5 2018/09/09 20:33:28 christos Exp $") #endif int asprintf(char **ptr, const char *fmt, ...) diff --git a/src/buffer.c b/src/buffer.c index 5f76b80d282c..6d8967d2fa73 100644 --- a/src/buffer.c +++ b/src/buffer.c @@ -27,7 +27,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: buffer.c,v 1.4 2018/02/21 21:26:00 christos Exp $") +FILE_RCSID("@(#)$File: buffer.c,v 1.6 2019/05/07 02:27:11 christos Exp $") #endif /* lint */ #include "magic.h" @@ -37,10 +37,13 @@ FILE_RCSID("@(#)$File: buffer.c,v 1.4 2018/02/21 21:26:00 christos Exp $") #include <sys/stat.h> void -buffer_init(struct buffer *b, int fd, const void *data, size_t len) +buffer_init(struct buffer *b, int fd, const struct stat *st, const void *data, + size_t len) { b->fd = fd; - if (b->fd == -1 || fstat(b->fd, &b->st) == -1) + if (st) + memcpy(&b->st, st, sizeof(b->st)); + else if (b->fd == -1 || fstat(b->fd, &b->st) == -1) memset(&b->st, 0, sizeof(b->st)); b->fbuf = data; b->flen = len; @@ -61,13 +64,13 @@ buffer_fill(const struct buffer *bb) struct buffer *b = CCAST(struct buffer *, bb); if (b->elen != 0) - return b->elen == (size_t)~0 ? -1 : 0; + return b->elen == CAST(size_t, ~0) ? -1 : 0; if (!S_ISREG(b->st.st_mode)) goto out; - b->elen = (size_t)b->st.st_size < b->flen ? - (size_t)b->st.st_size : b->flen; + b->elen = CAST(size_t, b->st.st_size) < b->flen ? + CAST(size_t, b->st.st_size) : b->flen; if ((b->ebuf = malloc(b->elen)) == NULL) goto out; @@ -79,6 +82,6 @@ buffer_fill(const struct buffer *bb) return 0; out: - b->elen = (size_t)~0; + b->elen = CAST(size_t, ~0); return -1; } diff --git a/src/cdf.c b/src/cdf.c index aad68cd0a33c..556a3ff86809 100644 --- a/src/cdf.c +++ b/src/cdf.c @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: cdf.c,v 1.110 2017/12/19 00:21:21 christos Exp $") +FILE_RCSID("@(#)$File: cdf.c,v 1.114 2019/02/20 02:35:27 christos Exp $") #endif #include <assert.h> @@ -47,9 +47,7 @@ FILE_RCSID("@(#)$File: cdf.c,v 1.110 2017/12/19 00:21:21 christos Exp $") #include <string.h> #include <time.h> #include <ctype.h> -#ifdef HAVE_LIMITS_H #include <limits.h> -#endif #ifndef EFTYPE #define EFTYPE EINVAL @@ -68,11 +66,14 @@ static union { uint32_t u; } cdf_bo; -#define NEED_SWAP (cdf_bo.u == (uint32_t)0x01020304) +#define NEED_SWAP (cdf_bo.u == CAST(uint32_t, 0x01020304)) -#define CDF_TOLE8(x) ((uint64_t)(NEED_SWAP ? _cdf_tole8(x) : (uint64_t)(x))) -#define CDF_TOLE4(x) ((uint32_t)(NEED_SWAP ? _cdf_tole4(x) : (uint32_t)(x))) -#define CDF_TOLE2(x) ((uint16_t)(NEED_SWAP ? _cdf_tole2(x) : (uint16_t)(x))) +#define CDF_TOLE8(x) \ + (CAST(uint64_t, NEED_SWAP ? _cdf_tole8(x) : CAST(uint64_t, x))) +#define CDF_TOLE4(x) \ + (CAST(uint32_t, NEED_SWAP ? _cdf_tole4(x) : CAST(uint32_t, x))) +#define CDF_TOLE2(x) \ + (CAST(uint16_t, NEED_SWAP ? _cdf_tole2(x) : CAST(uint16_t, x))) #define CDF_TOLE(x) (/*CONSTCOND*/sizeof(x) == 2 ? \ CDF_TOLE2(CAST(uint16_t, x)) : \ (/*CONSTCOND*/sizeof(x) == 4 ? \ @@ -90,7 +91,8 @@ static void * cdf_malloc(const char *file __attribute__((__unused__)), size_t line __attribute__((__unused__)), size_t n) { - DPRINTF(("%s,%zu: %s %zu\n", file, line, __func__, n)); + DPRINTF(("%s,%" SIZE_T_FORMAT "u: %s %" SIZE_T_FORMAT "u\n", + file, line, __func__, n)); return malloc(n); } @@ -99,7 +101,8 @@ static void * cdf_realloc(const char *file __attribute__((__unused__)), size_t line __attribute__((__unused__)), void *p, size_t n) { - DPRINTF(("%s,%zu: %s %zu\n", file, line, __func__, n)); + DPRINTF(("%s,%" SIZE_T_FORMAT "u: %s %" SIZE_T_FORMAT "u\n", + file, line, __func__, n)); return realloc(p, n); } @@ -108,7 +111,8 @@ static void * cdf_calloc(const char *file __attribute__((__unused__)), size_t line __attribute__((__unused__)), size_t n, size_t u) { - DPRINTF(("%s,%zu: %s %zu %zu\n", file, line, __func__, n, u)); + DPRINTF(("%s,%" SIZE_T_FORMAT "u: %s %" SIZE_T_FORMAT "u %" + SIZE_T_FORMAT "u\n", file, line, __func__, n, u)); return calloc(n, u); } @@ -119,8 +123,8 @@ static uint16_t _cdf_tole2(uint16_t sv) { uint16_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); d[0] = s[1]; d[1] = s[0]; return rv; @@ -133,8 +137,8 @@ static uint32_t _cdf_tole4(uint32_t sv) { uint32_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); d[0] = s[3]; d[1] = s[2]; d[2] = s[1]; @@ -149,8 +153,8 @@ static uint64_t _cdf_tole8(uint64_t sv) { uint64_t rv; - uint8_t *s = (uint8_t *)(void *)&sv; - uint8_t *d = (uint8_t *)(void *)&rv; + uint8_t *s = RCAST(uint8_t *, RCAST(void *, &sv)); + uint8_t *d = RCAST(uint8_t *, RCAST(void *, &rv)); d[0] = s[7]; d[1] = s[6]; d[2] = s[5]; @@ -215,15 +219,17 @@ cdf_swap_header(cdf_header_t *h) h->h_min_size_standard_stream = CDF_TOLE4(h->h_min_size_standard_stream); h->h_secid_first_sector_in_short_sat = - CDF_TOLE4((uint32_t)h->h_secid_first_sector_in_short_sat); + CDF_TOLE4(CAST(uint32_t, h->h_secid_first_sector_in_short_sat)); h->h_num_sectors_in_short_sat = CDF_TOLE4(h->h_num_sectors_in_short_sat); h->h_secid_first_sector_in_master_sat = - CDF_TOLE4((uint32_t)h->h_secid_first_sector_in_master_sat); + CDF_TOLE4(CAST(uint32_t, h->h_secid_first_sector_in_master_sat)); h->h_num_sectors_in_master_sat = CDF_TOLE4(h->h_num_sectors_in_master_sat); - for (i = 0; i < __arraycount(h->h_master_sat); i++) - h->h_master_sat[i] = CDF_TOLE4((uint32_t)h->h_master_sat[i]); + for (i = 0; i < __arraycount(h->h_master_sat); i++) { + h->h_master_sat[i] = + CDF_TOLE4(CAST(uint32_t, h->h_master_sat[i])); + } } void @@ -256,15 +262,16 @@ void cdf_swap_dir(cdf_directory_t *d) { d->d_namelen = CDF_TOLE2(d->d_namelen); - d->d_left_child = CDF_TOLE4((uint32_t)d->d_left_child); - d->d_right_child = CDF_TOLE4((uint32_t)d->d_right_child); - d->d_storage = CDF_TOLE4((uint32_t)d->d_storage); + d->d_left_child = CDF_TOLE4(CAST(uint32_t, d->d_left_child)); + d->d_right_child = CDF_TOLE4(CAST(uint32_t, d->d_right_child)); + d->d_storage = CDF_TOLE4(CAST(uint32_t, d->d_storage)); d->d_storage_uuid[0] = CDF_TOLE8(d->d_storage_uuid[0]); d->d_storage_uuid[1] = CDF_TOLE8(d->d_storage_uuid[1]); d->d_flags = CDF_TOLE4(d->d_flags); - d->d_created = CDF_TOLE8((uint64_t)d->d_created); - d->d_modified = CDF_TOLE8((uint64_t)d->d_modified); - d->d_stream_first_sector = CDF_TOLE4((uint32_t)d->d_stream_first_sector); + d->d_created = CDF_TOLE8(CAST(uint64_t, d->d_created)); + d->d_modified = CDF_TOLE8(CAST(uint64_t, d->d_modified)); + d->d_stream_first_sector = CDF_TOLE4( + CAST(uint32_t, d->d_stream_first_sector)); d->d_size = CDF_TOLE4(d->d_size); } @@ -321,11 +328,11 @@ static int cdf_check_stream_offset(const cdf_stream_t *sst, const cdf_header_t *h, const void *p, size_t tail, int line) { - const char *b = (const char *)sst->sst_tab; - const char *e = ((const char *)p) + tail; + const char *b = RCAST(const char *, sst->sst_tab); + const char *e = RCAST(const char *, p) + tail; size_t ss = cdf_check_stream(sst, h); /*LINTED*/(void)&line; - if (e >= b && (size_t)(e - b) <= ss * sst->sst_len) + if (e >= b && CAST(size_t, e - b) <= ss * sst->sst_len) return 0; DPRINTF(("%d: offset begin %p < end %p || %" SIZE_T_FORMAT "u" " > %" SIZE_T_FORMAT "u [%" SIZE_T_FORMAT "u %" @@ -338,23 +345,23 @@ cdf_check_stream_offset(const cdf_stream_t *sst, const cdf_header_t *h, static ssize_t cdf_read(const cdf_info_t *info, off_t off, void *buf, size_t len) { - size_t siz = (size_t)off + len; + size_t siz = CAST(size_t, off + len); - if ((off_t)(off + len) != (off_t)siz) + if (CAST(off_t, off + len) != CAST(off_t, siz)) goto out; if (info->i_buf != NULL && info->i_len >= siz) { (void)memcpy(buf, &info->i_buf[off], len); - return (ssize_t)len; + return CAST(ssize_t, len); } if (info->i_fd == -1) goto out; - if (pread(info->i_fd, buf, len, off) != (ssize_t)len) + if (pread(info->i_fd, buf, len, off) != CAST(ssize_t, len)) return -1; - return (ssize_t)len; + return CAST(ssize_t, len); out: errno = EINVAL; return -1; @@ -366,7 +373,7 @@ cdf_read_header(const cdf_info_t *info, cdf_header_t *h) char buf[512]; (void)memcpy(cdf_bo.s, "\01\02\03\04", 4); - if (cdf_read(info, (off_t)0, buf, sizeof(buf)) == -1) + if (cdf_read(info, CAST(off_t, 0), buf, sizeof(buf)) == -1) return -1; cdf_unpack_header(h, buf); cdf_swap_header(h); @@ -400,7 +407,7 @@ cdf_read_sector(const cdf_info_t *info, void *buf, size_t offs, size_t len, size_t ss = CDF_SEC_SIZE(h); size_t pos = CDF_SEC_POS(h, id); assert(ss == len); - return cdf_read(info, (off_t)pos, ((char *)buf) + offs, len); + return cdf_read(info, CAST(off_t, pos), RCAST(char *, buf) + offs, len); } ssize_t @@ -416,8 +423,8 @@ cdf_read_short_sector(const cdf_stream_t *sst, void *buf, size_t offs, pos + len, CDF_SEC_SIZE(h) * sst->sst_len)); goto out; } - (void)memcpy(((char *)buf) + offs, - ((const char *)sst->sst_tab) + pos, len); + (void)memcpy(RCAST(char *, buf) + offs, + RCAST(const char *, sst->sst_tab) + pos, len); return len; out: errno = EFTYPE; @@ -460,7 +467,7 @@ cdf_read_sat(const cdf_info_t *info, cdf_header_t *h, cdf_sat_t *sat) if (h->h_master_sat[i] < 0) break; if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, - h->h_master_sat[i]) != (ssize_t)ss) { + h->h_master_sat[i]) != CAST(ssize_t, ss)) { DPRINTF(("Reading sector %d", h->h_master_sat[i])); goto out1; } @@ -477,27 +484,29 @@ cdf_read_sat(const cdf_info_t *info, cdf_header_t *h, cdf_sat_t *sat) DPRINTF(("Reading master sector loop limit")); goto out3; } - if (cdf_read_sector(info, msa, 0, ss, h, mid) != (ssize_t)ss) { + if (cdf_read_sector(info, msa, 0, ss, h, mid) != + CAST(ssize_t, ss)) { DPRINTF(("Reading master sector %d", mid)); goto out2; } for (k = 0; k < nsatpersec; k++, i++) { - sec = CDF_TOLE4((uint32_t)msa[k]); + sec = CDF_TOLE4(CAST(uint32_t, msa[k])); if (sec < 0) goto out; if (i >= sat->sat_len) { - DPRINTF(("Out of bounds reading MSA %" SIZE_T_FORMAT - "u >= %" SIZE_T_FORMAT "u", i, sat->sat_len)); + DPRINTF(("Out of bounds reading MSA %" + SIZE_T_FORMAT "u >= %" SIZE_T_FORMAT "u", + i, sat->sat_len)); goto out3; } if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, - sec) != (ssize_t)ss) { + sec) != CAST(ssize_t, ss)) { DPRINTF(("Reading sector %d", CDF_TOLE4(msa[k]))); goto out2; } } - mid = CDF_TOLE4((uint32_t)msa[nsatpersec]); + mid = CDF_TOLE4(CAST(uint32_t, msa[nsatpersec])); } out: sat->sat_len = i; @@ -516,7 +525,7 @@ size_t cdf_count_chain(const cdf_sat_t *sat, cdf_secid_t sid, size_t size) { size_t i, j; - cdf_secid_t maxsector = (cdf_secid_t)((sat->sat_len * size) + cdf_secid_t maxsector = CAST(cdf_secid_t, (sat->sat_len * size) / sizeof(maxsector)); DPRINTF(("Chain:")); @@ -536,7 +545,7 @@ cdf_count_chain(const cdf_sat_t *sat, cdf_secid_t sid, size_t size) DPRINTF(("Sector %d >= %d\n", sid, maxsector)); goto out; } - sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); + sid = CDF_TOLE4(CAST(uint32_t, sat->sat_tab[sid])); } if (i == 0) { DPRINTF((" none, sid: %d\n", sid)); @@ -547,7 +556,7 @@ cdf_count_chain(const cdf_sat_t *sat, cdf_secid_t sid, size_t size) return i; out: errno = EFTYPE; - return (size_t)-1; + return CAST(size_t, -1); } int @@ -564,7 +573,7 @@ cdf_read_long_sector_chain(const cdf_info_t *info, const cdf_header_t *h, if (sid == CDF_SECID_END_OF_CHAIN || len == 0) return cdf_zero_stream(scn); - if (scn->sst_len == (size_t)-1) + if (scn->sst_len == CAST(size_t, -1)) goto out; scn->sst_tab = CDF_CALLOC(scn->sst_len, ss); @@ -583,7 +592,7 @@ cdf_read_long_sector_chain(const cdf_info_t *info, const cdf_header_t *h, goto out; } if ((nr = cdf_read_sector(info, scn->sst_tab, i * ss, ss, h, - sid)) != (ssize_t)ss) { + sid)) != CAST(ssize_t, ss)) { if (i == scn->sst_len - 1 && nr > 0) { /* Last sector might be truncated */ return 0; @@ -591,7 +600,7 @@ cdf_read_long_sector_chain(const cdf_info_t *info, const cdf_header_t *h, DPRINTF(("Reading long sector chain %d", sid)); goto out; } - sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); + sid = CDF_TOLE4(CAST(uint32_t, sat->sat_tab[sid])); } return 0; out: @@ -610,7 +619,7 @@ cdf_read_short_sector_chain(const cdf_header_t *h, scn->sst_dirlen = len; scn->sst_ss = ss; - if (scn->sst_len == (size_t)-1) + if (scn->sst_len == CAST(size_t, -1)) goto out; scn->sst_tab = CDF_CALLOC(scn->sst_len, ss); @@ -629,11 +638,11 @@ cdf_read_short_sector_chain(const cdf_header_t *h, goto out; } if (cdf_read_short_sector(sst, scn->sst_tab, i * ss, ss, h, - sid) != (ssize_t)ss) { + sid) != CAST(ssize_t, ss)) { DPRINTF(("Reading short sector chain %d", sid)); goto out; } - sid = CDF_TOLE4((uint32_t)ssat->sat_tab[sid]); + sid = CDF_TOLE4(CAST(uint32_t, ssat->sat_tab[sid])); } return 0; out: @@ -664,7 +673,7 @@ cdf_read_dir(const cdf_info_t *info, const cdf_header_t *h, cdf_secid_t sid = h->h_secid_first_directory; ns = cdf_count_chain(sat, sid, ss); - if (ns == (size_t)-1) + if (ns == CAST(size_t, -1)) return -1; nd = ss / CDF_DIRECTORY_SIZE; @@ -685,7 +694,8 @@ cdf_read_dir(const cdf_info_t *info, const cdf_header_t *h, DPRINTF(("Read dir loop limit")); goto out; } - if (cdf_read_sector(info, buf, 0, ss, h, sid) != (ssize_t)ss) { + if (cdf_read_sector(info, buf, 0, ss, h, sid) != + CAST(ssize_t, ss)) { DPRINTF(("Reading directory sector %d", sid)); goto out; } @@ -693,7 +703,7 @@ cdf_read_dir(const cdf_info_t *info, const cdf_header_t *h, cdf_unpack_dir(&dir->dir_tab[i * nd + j], &buf[j * CDF_DIRECTORY_SIZE]); } - sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); + sid = CDF_TOLE4(CAST(uint32_t, sat->sat_tab[sid])); } if (NEED_SWAP) for (i = 0; i < dir->dir_len; i++) @@ -718,7 +728,7 @@ cdf_read_ssat(const cdf_info_t *info, const cdf_header_t *h, ssat->sat_tab = NULL; ssat->sat_len = cdf_count_chain(sat, sid, ss); - if (ssat->sat_len == (size_t)-1) + if (ssat->sat_len == CAST(size_t, -1)) goto out; ssat->sat_tab = CAST(cdf_secid_t *, CDF_CALLOC(ssat->sat_len, ss)); @@ -737,11 +747,11 @@ cdf_read_ssat(const cdf_info_t *info, const cdf_header_t *h, goto out; } if (cdf_read_sector(info, ssat->sat_tab, i * ss, ss, h, sid) != - (ssize_t)ss) { + CAST(ssize_t, ss)) { DPRINTF(("Reading short sat sector %d", sid)); goto out1; } - sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); + sid = CDF_TOLE4(CAST(uint32_t, sat->sat_tab[sid])); } return 0; out: @@ -791,7 +801,7 @@ cdf_namecmp(const char *d, const uint16_t *s, size_t l) { for (; l--; d++, s++) if (*d != CDF_TOLE2(*s)) - return (unsigned char)*d - CDF_TOLE2(*s); + return CAST(unsigned char, *d) - CDF_TOLE2(*s); return 0; } @@ -859,7 +869,7 @@ cdf_offset(const void *p, size_t l) } static const uint8_t * -cdf_get_property_info_pos(const cdf_stream_t *sst, const cdf_header_t *h, +cdf_get_property_info_pos(const cdf_stream_t *sst, const cdf_header_t *h, const uint8_t *p, const uint8_t *e, size_t i) { size_t tail = (i << 1) + 1; @@ -874,7 +884,7 @@ cdf_get_property_info_pos(const cdf_stream_t *sst, const cdf_header_t *h, __LINE__) == -1) return NULL; ofs = CDF_GETUINT32(p, tail); - q = CAST(const uint8_t *, cdf_offset(CAST(const void *, p), + q = CAST(const uint8_t *, cdf_offset(CAST(const void *, p), ofs - 2 * sizeof(uint32_t))); if (q < p) { @@ -896,8 +906,8 @@ cdf_grow_info(cdf_property_info_t **info, size_t *maxcount, size_t incr) size_t newcount = *maxcount + incr; if (newcount > CDF_PROP_LIMIT) { - DPRINTF(("exceeded property limit %zu > %zu\n", - newcount, CDF_PROP_LIMIT)); + DPRINTF(("exceeded property limit %" SIZE_T_FORMAT "u > %" + SIZE_T_FORMAT "u\n", newcount, CDF_PROP_LIMIT)); goto out; } inp = CAST(cdf_property_info_t *, @@ -922,7 +932,7 @@ cdf_copy_info(cdf_property_info_t *inp, const void *p, const void *e, if (inp->pi_type & CDF_VECTOR) return 0; - if ((size_t)(CAST(const char *, e) - CAST(const char *, p)) < len) + if (CAST(size_t, CAST(const char *, e) - CAST(const char *, p)) < len) return 0; (void)memcpy(&inp->pi_val, p, len); @@ -1065,10 +1075,10 @@ cdf_read_property_info(const cdf_stream_t *sst, const cdf_header_t *h, inp[i].pi_str.s_buf = CAST(const char *, CAST(const void *, &q[o4])); - DPRINTF(("o=%zu l=%d(%" SIZE_T_FORMAT - "u), t=%zu s=%s\n", o4, l, - CDF_ROUND(l, sizeof(l)), left, - inp[i].pi_str.s_buf)); + DPRINTF(("o=%" SIZE_T_FORMAT "u l=%d(%" + SIZE_T_FORMAT "u), t=%" SIZE_T_FORMAT + "u s=%s\n", o4, l, CDF_ROUND(l, sizeof(l)), + left, inp[i].pi_str.s_buf)); if (l & 1) l++; @@ -1108,8 +1118,9 @@ cdf_unpack_summary_info(const cdf_stream_t *sst, const cdf_header_t *h, const cdf_summary_info_header_t *si = CAST(const cdf_summary_info_header_t *, sst->sst_tab); const cdf_section_declaration_t *sd = - CAST(const cdf_section_declaration_t *, (const void *) - ((const char *)sst->sst_tab + CDF_SECTION_DECLARATION_OFFSET)); + CAST(const cdf_section_declaration_t *, RCAST(const void *, + RCAST(const char *, sst->sst_tab) + + CDF_SECTION_DECLARATION_OFFSET)); if (cdf_check_stream_offset(sst, h, si, sizeof(*si), __LINE__) == -1 || cdf_check_stream_offset(sst, h, sd, sizeof(*sd), __LINE__) == -1) @@ -1260,28 +1271,28 @@ cdf_print_elapsed_time(char *buf, size_t bufsiz, cdf_timestamp_t ts) int days, hours, mins, secs; ts /= CDF_TIME_PREC; - secs = (int)(ts % 60); + secs = CAST(int, ts % 60); ts /= 60; - mins = (int)(ts % 60); + mins = CAST(int, ts % 60); ts /= 60; - hours = (int)(ts % 24); + hours = CAST(int, ts % 24); ts /= 24; - days = (int)ts; + days = CAST(int, ts); if (days) { len += snprintf(buf + len, bufsiz - len, "%dd+", days); - if ((size_t)len >= bufsiz) + if (CAST(size_t, len) >= bufsiz) return len; } if (days || hours) { len += snprintf(buf + len, bufsiz - len, "%.2d:", hours); - if ((size_t)len >= bufsiz) + if (CAST(size_t, len) >= bufsiz) return len; } len += snprintf(buf + len, bufsiz - len, "%.2d:", mins); - if ((size_t)len >= bufsiz) + if (CAST(size_t, len) >= bufsiz) return len; len += snprintf(buf + len, bufsiz - len, "%.2d", secs); @@ -1293,7 +1304,7 @@ cdf_u16tos8(char *buf, size_t len, const uint16_t *p) { size_t i; for (i = 0; i < len && p[i]; i++) - buf[i] = (char)p[i]; + buf[i] = CAST(char, p[i]); buf[i] = '\0'; return buf; } diff --git a/src/cdf.h b/src/cdf.h index f2df8306b17d..2f7e554b7180 100644 --- a/src/cdf.h +++ b/src/cdf.h @@ -76,9 +76,9 @@ typedef struct { cdf_secid_t h_master_sat[436/4]; } cdf_header_t; -#define CDF_SEC_SIZE(h) ((size_t)(1 << (h)->h_sec_size_p2)) +#define CDF_SEC_SIZE(h) CAST(size_t, 1 << (h)->h_sec_size_p2) #define CDF_SEC_POS(h, secid) (CDF_SEC_SIZE(h) + (secid) * CDF_SEC_SIZE(h)) -#define CDF_SHORT_SEC_SIZE(h) ((size_t)(1 << (h)->h_short_sec_size_p2)) +#define CDF_SHORT_SEC_SIZE(h) CAST(size_t, 1 << (h)->h_short_sec_size_p2) #define CDF_SHORT_SEC_POS(h, secid) ((secid) * CDF_SHORT_SEC_SIZE(h)) typedef int32_t cdf_dirid_t; @@ -272,7 +272,7 @@ typedef struct { typedef struct { uint16_t ce_namlen; uint32_t ce_num; - uint64_t ce_timestamp; + uint64_t ce_timestamp; uint16_t ce_name[256]; } cdf_catalog_entry_t; diff --git a/src/cdf_time.c b/src/cdf_time.c index 2bdcd2a7f7a8..e4eea4c737f3 100644 --- a/src/cdf_time.c +++ b/src/cdf_time.c @@ -27,7 +27,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: cdf_time.c,v 1.16 2017/03/29 15:57:48 christos Exp $") +FILE_RCSID("@(#)$File: cdf_time.c,v 1.19 2019/03/12 20:43:05 christos Exp $") #endif #include <time.h> @@ -56,7 +56,7 @@ cdf_getdays(int year) for (y = CDF_BASE_YEAR; y < year; y++) days += isleap(y) + 365; - + return days; } @@ -68,7 +68,7 @@ cdf_getday(int year, int days) { size_t m; - for (m = 0; m < sizeof(mdays) / sizeof(mdays[0]); m++) { + for (m = 0; m < __arraycount(mdays); m++) { int sub = mdays[m] + (m == 1 && isleap(year)); if (days < sub) return days; @@ -77,7 +77,7 @@ cdf_getday(int year, int days) return days; } -/* +/* * Return the 0...11 month number. */ static int @@ -85,14 +85,14 @@ cdf_getmonth(int year, int days) { size_t m; - for (m = 0; m < sizeof(mdays) / sizeof(mdays[0]); m++) { + for (m = 0; m < __arraycount(mdays); m++) { days -= mdays[m]; if (m == 1 && isleap(year)) days--; if (days <= 0) - return (int)m; + return CAST(int, m); } - return (int)m; + return CAST(int, m); } int @@ -108,22 +108,22 @@ cdf_timestamp_to_timespec(struct timespec *ts, cdf_timestamp_t t) ts->tv_nsec = (t % CDF_TIME_PREC) * 100; t /= CDF_TIME_PREC; - tm.tm_sec = (int)(t % 60); + tm.tm_sec = CAST(int, t % 60); t /= 60; - tm.tm_min = (int)(t % 60); + tm.tm_min = CAST(int, t % 60); t /= 60; - tm.tm_hour = (int)(t % 24); + tm.tm_hour = CAST(int, t % 24); t /= 24; /* XXX: Approx */ - tm.tm_year = (int)(CDF_BASE_YEAR + (t / 365)); + tm.tm_year = CAST(int, CDF_BASE_YEAR + (t / 365)); rdays = cdf_getdays(tm.tm_year); t -= rdays - 1; - tm.tm_mday = cdf_getday(tm.tm_year, (int)t); - tm.tm_mon = cdf_getmonth(tm.tm_year, (int)t); + tm.tm_mday = cdf_getday(tm.tm_year, CAST(int, t)); + tm.tm_mon = cdf_getmonth(tm.tm_year, CAST(int, t)); tm.tm_wday = 0; tm.tm_yday = 0; tm.tm_isdst = 0; @@ -172,7 +172,7 @@ cdf_ctime(const time_t *sec, char *buf) if (ptr != NULL) return buf; (void)snprintf(buf, 26, "*Bad* %#16.16" INT64_T_FORMAT "x\n", - (long long)*sec); + CAST(long long, *sec)); return buf; } diff --git a/src/compress.c b/src/compress.c index 5d565d56089e..95e42a24dd52 100644 --- a/src/compress.c +++ b/src/compress.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -29,13 +29,13 @@ * compress routines: * zmagic() - returns 0 if not recognized, uncompresses and prints * information if recognized - * uncompress(method, old, n, newch) - uncompress old into new, + * uncompress(method, old, n, newch) - uncompress old into new, * using method, return sizeof new */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: compress.c,v 1.107 2018/04/28 18:48:22 christos Exp $") +FILE_RCSID("@(#)$File: compress.c,v 1.121 2019/05/07 02:27:11 christos Exp $") #endif #include "magic.h" @@ -47,12 +47,10 @@ FILE_RCSID("@(#)$File: compress.c,v 1.107 2018/04/28 18:48:22 christos Exp $") #include <errno.h> #include <ctype.h> #include <stdarg.h> -#ifdef HAVE_SIGNAL_H #include <signal.h> -# ifndef HAVE_SIG_T +#ifndef HAVE_SIG_T typedef void (*sig_t)(int); -# endif /* HAVE_SIG_T */ -#endif +#endif /* HAVE_SIG_T */ #if !defined(__MINGW32__) && !defined(WIN32) #include <sys/ioctl.h> #endif @@ -62,10 +60,17 @@ typedef void (*sig_t)(int); #if defined(HAVE_SYS_TIME_H) #include <sys/time.h> #endif + #if defined(HAVE_ZLIB_H) && defined(ZLIBSUPPORT) #define BUILTIN_DECOMPRESS #include <zlib.h> #endif + +#if defined(HAVE_BZLIB_H) +#define BUILTIN_BZLIB +#include <bzlib.h> +#endif + #ifdef DEBUG int tty = -1; #define DPRINTF(...) do { \ @@ -137,30 +142,34 @@ static const char *zstd_args[] = { "zstd", "-cd", NULL }; +#define do_zlib NULL +#define do_bzlib NULL + private const struct { const void *magic; size_t maglen; const char **argv; + void *unused; } compr[] = { - { "\037\235", 2, gzip_args }, /* compressed */ + { "\037\235", 2, gzip_args, NULL }, /* compressed */ /* Uncompress can get stuck; so use gzip first if we have it * Idea from Damien Clark, thanks! */ - { "\037\235", 2, uncompress_args }, /* compressed */ - { "\037\213", 2, gzip_args }, /* gzipped */ - { "\037\236", 2, gzip_args }, /* frozen */ - { "\037\240", 2, gzip_args }, /* SCO LZH */ + { "\037\235", 2, uncompress_args, NULL }, /* compressed */ + { "\037\213", 2, gzip_args, do_zlib }, /* gzipped */ + { "\037\236", 2, gzip_args, NULL }, /* frozen */ + { "\037\240", 2, gzip_args, NULL }, /* SCO LZH */ /* the standard pack utilities do not accept standard input */ - { "\037\036", 2, gzip_args }, /* packed */ - { "PK\3\4", 4, gzip_args }, /* pkzipped, */ + { "\037\036", 2, gzip_args, NULL }, /* packed */ + { "PK\3\4", 4, gzip_args, NULL }, /* pkzipped, */ /* ...only first file examined */ - { "BZh", 3, bzip2_args }, /* bzip2-ed */ - { "LZIP", 4, lzip_args }, /* lzip-ed */ - { "\3757zXZ\0", 6, xz_args }, /* XZ Utils */ - { "LRZI", 4, lrzip_args }, /* LRZIP */ - { "\004\"M\030",4, lz4_args }, /* LZ4 */ - { "\x28\xB5\x2F\xFD", 4, zstd_args }, /* zstd */ + { "BZh", 3, bzip2_args, do_bzlib }, /* bzip2-ed */ + { "LZIP", 4, lzip_args, NULL }, /* lzip-ed */ + { "\3757zXZ\0", 6, xz_args, NULL }, /* XZ Utils */ + { "LRZI", 4, lrzip_args, NULL }, /* LRZIP */ + { "\004\"M\030",4, lz4_args, NULL }, /* LZ4 */ + { "\x28\xB5\x2F\xFD", 4, zstd_args, NULL }, /* zstd */ #ifdef ZLIBSUPPORT - { RCAST(const void *, zlibcmp), 0, zlib_args }, /* zlib */ + { RCAST(const void *, zlibcmp), 0, zlib_args, NULL }, /* zlib */ #endif }; @@ -170,7 +179,7 @@ private const struct { private ssize_t swrite(int, const void *, size_t); #if HAVE_FORK -private size_t ncompr = sizeof(compr) / sizeof(compr[0]); +private size_t ncompr = __arraycount(compr); private int uncompressbuf(int, size_t, size_t, const unsigned char *, unsigned char **, size_t *); #ifdef BUILTIN_DECOMPRESS @@ -179,6 +188,11 @@ private int uncompresszlib(const unsigned char *, unsigned char **, size_t, private int uncompressgzipped(const unsigned char *, unsigned char **, size_t, size_t *); #endif +#ifdef BUILTIN_BZLIB +private int uncompressbzlib(const unsigned char *, unsigned char **, size_t, + size_t *, int); +#endif + static int makeerror(unsigned char **, size_t *, const char *, ...) __attribute__((__format__(__printf__, 3, 4))); private const char *methodname(size_t); @@ -210,18 +224,14 @@ file_zmagic(struct magic_set *ms, const struct buffer *b, const char *name) int urv, prv, rv = 0; int mime = ms->flags & MAGIC_MIME; int fd = b->fd; - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; -#ifdef HAVE_SIGNAL_H - sig_t osigpipe; -#endif + int sa_saved = 0; + struct sigaction sig_act; if ((ms->flags & MAGIC_COMPRESS) == 0) return 0; -#ifdef HAVE_SIGNAL_H - osigpipe = signal(SIGPIPE, SIG_IGN); -#endif for (i = 0; i < ncompr; i++) { int zm; if (nbytes < compr[i].maglen) @@ -236,10 +246,20 @@ file_zmagic(struct magic_set *ms, const struct buffer *b, const char *name) if (!zm) continue; + + /* Prevent SIGPIPE death if child dies unexpectedly */ + if (!sa_saved) { + //We can use sig_act for both new and old, but + struct sigaction new_act; + memset(&new_act, 0, sizeof(new_act)); + new_act.sa_handler = SIG_IGN; + sa_saved = sigaction(SIGPIPE, &new_act, &sig_act) != -1; + } + nsz = nbytes; urv = uncompressbuf(fd, ms->bytes_max, i, buf, &newbuf, &nsz); - DPRINTF("uncompressbuf = %d, %s, %zu\n", urv, (char *)newbuf, - nsz); + DPRINTF("uncompressbuf = %d, %s, %" SIZE_T_FORMAT "u\n", urv, + (char *)newbuf, nsz); switch (urv) { case OKDATA: case ERRDATA: @@ -247,7 +267,7 @@ file_zmagic(struct magic_set *ms, const struct buffer *b, const char *name) if (urv == ERRDATA) prv = format_decompression_error(ms, i, newbuf); else - prv = file_buffer(ms, -1, name, newbuf, nsz); + prv = file_buffer(ms, -1, NULL, name, newbuf, nsz); if (prv == -1) goto error; rv = 1; @@ -264,8 +284,11 @@ file_zmagic(struct magic_set *ms, const struct buffer *b, const char *name) * XXX: If file_buffer fails here, we overwrite * the compressed text. FIXME. */ - if (file_buffer(ms, -1, NULL, buf, nbytes) == -1) + if (file_buffer(ms, -1, NULL, NULL, buf, nbytes) == -1) { + if (file_pop_buffer(ms, pb) != NULL) + abort(); goto error; + } if ((rbuf = file_pop_buffer(ms, pb)) != NULL) { if (file_printf(ms, "%s", rbuf) == -1) { free(rbuf); @@ -289,9 +312,9 @@ file_zmagic(struct magic_set *ms, const struct buffer *b, const char *name) out: DPRINTF("rv = %d\n", rv); -#ifdef HAVE_SIGNAL_H - (void)signal(SIGPIPE, osigpipe); -#endif + if (sa_saved && sig_act.sa_handler != SIG_IGN) + (void)sigaction(SIGPIPE, &sig_act, NULL); + free(newbuf); ms->flags |= MAGIC_COMPRESS; DPRINTF("Zmagic returns %d\n", rv); @@ -367,7 +390,7 @@ sread(int fd, void *buf, size_t n, int canbepipe __attribute__((__unused__))) (void)ioctl(fd, FIONREAD, &t); } - if (t > 0 && (size_t)t < n) { + if (t > 0 && CAST(size_t, t) < n) { n = t; rn = n; } @@ -411,7 +434,9 @@ file_pipe2file(struct magic_set *ms, int fd, const void *startbuf, #else { int te; + mode_t ou = umask(0); tfd = mkstemp(buf); + (void)umask(ou); te = errno; (void)unlink(buf); errno = te; @@ -423,11 +448,11 @@ file_pipe2file(struct magic_set *ms, int fd, const void *startbuf, return -1; } - if (swrite(tfd, startbuf, nbytes) != (ssize_t)nbytes) + if (swrite(tfd, startbuf, nbytes) != CAST(ssize_t, nbytes)) r = 1; else { while ((r = sread(fd, buf, sizeof(buf), 1)) > 0) - if (swrite(tfd, buf, (size_t)r) != r) + if (swrite(tfd, buf, CAST(size_t, r)) != r) break; } @@ -452,7 +477,7 @@ file_pipe2file(struct magic_set *ms, int fd, const void *startbuf, return -1; } (void)close(tfd); - if (lseek(fd, (off_t)0, SEEK_SET) == (off_t)-1) { + if (lseek(fd, CAST(off_t, 0), SEEK_SET) == CAST(off_t, -1)) { file_badseek(ms); return -1; } @@ -509,7 +534,7 @@ uncompresszlib(const unsigned char *old, unsigned char **newch, int rc; z_stream z; - if ((*newch = CAST(unsigned char *, malloc(bytes_max + 1))) == NULL) + if ((*newch = CAST(unsigned char *, malloc(bytes_max + 1))) == NULL) return makeerror(newch, n, "No buffer, %s", strerror(errno)); z.next_in = CCAST(Bytef *, old); @@ -529,18 +554,18 @@ uncompresszlib(const unsigned char *old, unsigned char **newch, if (rc != Z_OK && rc != Z_STREAM_END) goto err; - *n = (size_t)z.total_out; + *n = CAST(size_t, z.total_out); rc = inflateEnd(&z); if (rc != Z_OK) goto err; - + /* let's keep the nul-terminate tradition */ (*newch)[*n] = '\0'; return OKDATA; err: - strlcpy((char *)*newch, z.msg ? z.msg : zError(rc), bytes_max); - *n = strlen((char *)*newch); + strlcpy(RCAST(char *, *newch), z.msg ? z.msg : zError(rc), bytes_max); + *n = strlen(RCAST(char *, *newch)); return ERRDATA; } #endif @@ -560,7 +585,7 @@ makeerror(unsigned char **buf, size_t *len, const char *fmt, ...) *len = 0; return NODATA; } - *buf = (unsigned char *)msg; + *buf = RCAST(unsigned char *, msg); *len = strlen(msg); return ERRDATA; } @@ -582,52 +607,42 @@ closep(int *fd) closefd(fd, i); } -static void -copydesc(int i, int *fd) +static int +copydesc(int i, int fd) { - int j = fd[i == STDIN_FILENO ? 0 : 1]; - if (j == i) - return; - if (dup2(j, i) == -1) { - DPRINTF("dup(%d, %d) failed (%s)\n", j, i, strerror(errno)); + if (fd == i) + return 0; /* "no dup was necessary" */ + if (dup2(fd, i) == -1) { + DPRINTF("dup(%d, %d) failed (%s)\n", fd, i, strerror(errno)); exit(1); } - closep(fd); + return 1; } -static void -writechild(int fdp[3][2], const void *old, size_t n) +static pid_t +writechild(int fd, const void *old, size_t n) { - int status; + pid_t pid; - closefd(fdp[STDIN_FILENO], 0); - /* + /* * fork again, to avoid blocking because both * pipes filled */ - switch (fork()) { - case 0: /* child */ - closefd(fdp[STDOUT_FILENO], 0); - if (swrite(fdp[STDIN_FILENO][1], old, n) != (ssize_t)n) { - DPRINTF("Write failed (%s)\n", strerror(errno)); - exit(1); - } - exit(0); - /*NOTREACHED*/ - - case -1: + pid = fork(); + if (pid == -1) { DPRINTF("Fork failed (%s)\n", strerror(errno)); exit(1); - /*NOTREACHED*/ - - default: /* parent */ - if (wait(&status) == -1) { - DPRINTF("Wait failed (%s)\n", strerror(errno)); + } + if (pid == 0) { + /* child */ + if (swrite(fd, old, n) != CAST(ssize_t, n)) { + DPRINTF("Write failed (%s)\n", strerror(errno)); exit(1); } - DPRINTF("Grandchild wait return %#x\n", status); + exit(0); } - closefd(fdp[STDIN_FILENO], 1); + /* parent */ + return pid; } static ssize_t @@ -637,17 +652,17 @@ filter_error(unsigned char *ubuf, ssize_t n) char *buf; ubuf[n] = '\0'; - buf = (char *)ubuf; - while (isspace((unsigned char)*buf)) + buf = RCAST(char *, ubuf); + while (isspace(CAST(unsigned char, *buf))) buf++; DPRINTF("Filter error[[[%s]]]\n", buf); - if ((p = strchr((char *)buf, '\n')) != NULL) + if ((p = strchr(CAST(char *, buf), '\n')) != NULL) *p = '\0'; - if ((p = strchr((char *)buf, ';')) != NULL) + if ((p = strchr(CAST(char *, buf), ';')) != NULL) *p = '\0'; - if ((p = strrchr((char *)buf, ':')) != NULL) { + if ((p = strrchr(CAST(char *, buf), ':')) != NULL) { ++p; - while (isspace((unsigned char)*p)) + while (isspace(CAST(unsigned char, *p))) p++; n = strlen(p); memmove(ubuf, p, CAST(size_t, n + 1)); @@ -674,7 +689,9 @@ uncompressbuf(int fd, size_t bytes_max, size_t method, const unsigned char *old, unsigned char **newch, size_t* n) { int fdp[3][2]; - int status, rv; + int status, rv, w; + pid_t pid; + pid_t writepid = -1; size_t i; ssize_t r; @@ -698,43 +715,68 @@ uncompressbuf(int fd, size_t bytes_max, size_t method, const unsigned char *old, return makeerror(newch, n, "Cannot create pipe, %s", strerror(errno)); } - switch (fork()) { - case 0: /* child */ + + /* For processes with large mapped virtual sizes, vfork + * may be _much_ faster (10-100 times) than fork. + */ + pid = vfork(); + if (pid == -1) { + return makeerror(newch, n, "Cannot vfork, %s", + strerror(errno)); + } + if (pid == 0) { + /* child */ + /* Note: we are after vfork, do not modify memory + * in a way which confuses parent. In particular, + * do not modify fdp[i][j]. + */ if (fd != -1) { - fdp[STDIN_FILENO][0] = fd; - (void) lseek(fd, (off_t)0, SEEK_SET); + (void) lseek(fd, CAST(off_t, 0), SEEK_SET); + if (copydesc(STDIN_FILENO, fd)) + (void) close(fd); + } else { + if (copydesc(STDIN_FILENO, fdp[STDIN_FILENO][0])) + (void) close(fdp[STDIN_FILENO][0]); + if (fdp[STDIN_FILENO][1] > 2) + (void) close(fdp[STDIN_FILENO][1]); } - - for (i = 0; i < __arraycount(fdp); i++) - copydesc(CAST(int, i), fdp[i]); +///FIXME: if one of the fdp[i][j] is 0 or 1, this can bomb spectacularly + if (copydesc(STDOUT_FILENO, fdp[STDOUT_FILENO][1])) + (void) close(fdp[STDOUT_FILENO][1]); + if (fdp[STDOUT_FILENO][0] > 2) + (void) close(fdp[STDOUT_FILENO][0]); + + if (copydesc(STDERR_FILENO, fdp[STDERR_FILENO][1])) + (void) close(fdp[STDERR_FILENO][1]); + if (fdp[STDERR_FILENO][0] > 2) + (void) close(fdp[STDERR_FILENO][0]); (void)execvp(compr[method].argv[0], - (char *const *)(intptr_t)compr[method].argv); - dprintf(STDERR_FILENO, "exec `%s' failed, %s", + RCAST(char *const *, RCAST(intptr_t, compr[method].argv))); + dprintf(STDERR_FILENO, "exec `%s' failed, %s", compr[method].argv[0], strerror(errno)); - exit(1); - /*NOTREACHED*/ - case -1: - return makeerror(newch, n, "Cannot fork, %s", - strerror(errno)); - - default: /* parent */ - for (i = 1; i < __arraycount(fdp); i++) - closefd(fdp[i], 1); - - /* Write the buffer data to the child, if we don't have fd */ - if (fd == -1) - writechild(fdp, old, *n); + _exit(1); /* _exit(), not exit(), because of vfork */ + } + /* parent */ + /* Close write sides of child stdout/err pipes */ + for (i = 1; i < __arraycount(fdp); i++) + closefd(fdp[i], 1); + /* Write the buffer data to child stdin, if we don't have fd */ + if (fd == -1) { + closefd(fdp[STDIN_FILENO], 0); + writepid = writechild(fdp[STDIN_FILENO][1], old, *n); + closefd(fdp[STDIN_FILENO], 1); + } - *newch = CAST(unsigned char *, malloc(bytes_max + 1)); - if (*newch == NULL) { - rv = makeerror(newch, n, "No buffer, %s", - strerror(errno)); - goto err; - } - rv = OKDATA; - if ((r = sread(fdp[STDOUT_FILENO][0], *newch, bytes_max, 0)) > 0) - break; + *newch = CAST(unsigned char *, malloc(bytes_max + 1)); + if (*newch == NULL) { + rv = makeerror(newch, n, "No buffer, %s", + strerror(errno)); + goto err; + } + rv = OKDATA; + r = sread(fdp[STDOUT_FILENO][0], *newch, bytes_max, 0); + if (r <= 0) { DPRINTF("Read stdout failed %d (%s)\n", fdp[STDOUT_FILENO][0], r != -1 ? strerror(errno) : "no data"); @@ -743,7 +785,7 @@ uncompressbuf(int fd, size_t bytes_max, size_t method, const unsigned char *old, (r = sread(fdp[STDERR_FILENO][0], *newch, bytes_max, 0)) > 0) { r = filter_error(*newch, r); - break; + goto ok; } free(*newch); if (r == 0) @@ -753,7 +795,7 @@ uncompressbuf(int fd, size_t bytes_max, size_t method, const unsigned char *old, rv = makeerror(newch, n, "No data"); goto err; } - +ok: *n = r; /* NUL terminate, as every buffer is handled here. */ (*newch)[*n] = '\0'; @@ -761,7 +803,10 @@ err: closefd(fdp[STDIN_FILENO], 1); closefd(fdp[STDOUT_FILENO], 0); closefd(fdp[STDERR_FILENO], 0); - if (wait(&status) == -1) { + + w = waitpid(pid, &status, 0); +wait_err: + if (w == -1) { free(*newch); rv = makeerror(newch, n, "Wait failed, %s", strerror(errno)); DPRINTF("Child wait return %#x\n", status); @@ -770,10 +815,19 @@ err: } else if (WEXITSTATUS(status) != 0) { DPRINTF("Child exited (%#x)\n", WEXITSTATUS(status)); } + if (writepid > 0) { + /* _After_ we know decompressor has exited, our input writer + * definitely will exit now (at worst, writing fails in it, + * since output fd is closed now on the reading size). + */ + w = waitpid(writepid, &status, 0); + writepid = -1; + goto wait_err; + } + + closefd(fdp[STDIN_FILENO], 0); //why? it is already closed here! + DPRINTF("Returning %p n=%" SIZE_T_FORMAT "u rv=%d\n", *newch, *n, rv); - closefd(fdp[STDIN_FILENO], 0); - DPRINTF("Returning %p n=%zu rv=%d\n", *newch, *n, rv); - return rv; } #endif diff --git a/src/der.c b/src/der.c index d017b63983f0..8867c568034d 100644 --- a/src/der.c +++ b/src/der.c @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: der.c,v 1.13 2018/06/23 15:15:26 christos Exp $") +FILE_RCSID("@(#)$File: der.c,v 1.16 2019/02/20 02:35:27 christos Exp $") #endif #endif @@ -56,7 +56,7 @@ FILE_RCSID("@(#)$File: der.c,v 1.13 2018/06/23 15:15:26 christos Exp $") #include <err.h> #endif -#define DER_BAD ((uint32_t)-1) +#define DER_BAD CAST(uint32_t, -1) #define DER_CLASS_UNIVERSAL 0 #define DER_CLASS_APPLICATION 1 @@ -207,7 +207,7 @@ getlength(const uint8_t *c, size_t *p, size_t l) static const char * der_tag(char *buf, size_t len, uint32_t tag) { - if (tag < DER_TAG_LONG) + if (tag < DER_TAG_LONG) strlcpy(buf, der__tag[tag], len); else snprintf(buf, len, "%#x", tag); @@ -224,11 +224,11 @@ der_data(char *buf, size_t blen, uint32_t tag, const void *q, uint32_t len) case DER_TAG_UTF8_STRING: case DER_TAG_IA5_STRING: case DER_TAG_UTCTIME: - return snprintf(buf, blen, "%.*s", len, (const char *)q); + return snprintf(buf, blen, "%.*s", len, RCAST(const char *, q)); default: break; } - + for (uint32_t i = 0; i < len; i++) { uint32_t z = i << 1; if (z < blen - 2) @@ -245,18 +245,21 @@ der_offs(struct magic_set *ms, struct magic *m, size_t nbytes) if (gettag(b, &offs, len) == DER_BAD) return -1; - DPRINTF(("%s1: %d %zu %u\n", __func__, ms->offset, offs, m->offset)); + DPRINTF(("%s1: %d %" SIZE_T_FORMAT "u %u\n", __func__, ms->offset, + offs, m->offset)); uint32_t tlen = getlength(b, &offs, len); if (tlen == DER_BAD) return -1; - DPRINTF(("%s2: %d %zu %u\n", __func__, ms->offset, offs, tlen)); + DPRINTF(("%s2: %d %" SIZE_T_FORMAT "u %u\n", __func__, ms->offset, + offs, tlen)); offs += ms->offset + m->offset; DPRINTF(("cont_level = %d\n", m->cont_level)); #ifdef DEBUG_DER for (size_t i = 0; i < m->cont_level; i++) - printf("cont_level[%zu] = %u\n", i, ms->c.li[i].off); + printf("cont_level[%" SIZE_T_FORMAT "u] = %u\n", i, + ms->c.li[i].off); #endif if (m->cont_level != 0) { if (offs + tlen > nbytes) @@ -304,22 +307,23 @@ again: s++; goto val; default: - if (!isdigit((unsigned char)*s)) + if (!isdigit(CAST(unsigned char, *s))) return 0; slen = 0; do slen = slen * 10 + *s - '0'; - while (isdigit((unsigned char)*++s)); + while (isdigit(CAST(unsigned char, *++s))); if ((ms->flags & MAGIC_DEBUG) != 0) - fprintf(stderr, "%s: len %zu %u\n", __func__, - slen, tlen); + fprintf(stderr, "%s: len %" SIZE_T_FORMAT "u %u\n", + __func__, slen, tlen); if (tlen != slen) return 0; goto again; } val: - DPRINTF(("%s: before data %zu %u\n", __func__, offs, tlen)); + DPRINTF(("%s: before data %" SIZE_T_FORMAT "u %u\n", __func__, offs, + tlen)); der_data(buf, sizeof(buf), tag, b + offs, tlen); if ((ms->flags & MAGIC_DEBUG) != 0) fprintf(stderr, "%s: data %s %s\n", __func__, buf, s); @@ -343,7 +347,7 @@ printtag(uint32_t tag, const void *q, uint32_t len) default: break; } - + for (uint32_t i = 0; i < len; i++) printf("%.2x", d[i]); printf("\n"); @@ -367,8 +371,9 @@ printdata(size_t level, const void *v, size_t x, size_t l) if (p + x >= ep) break; uint32_t len = getlength(p, &x, ep - p + x); - - printf("%zu %zu-%zu %c,%c,%s,%u:", level, ox, x, + + printf("%" SIZE_T_FORMAT "u %" SIZE_T_FORMAT "u-%" + SIZE_T_FORMAT "u %c,%c,%s,%u:", level, ox, x, der_class[c], der_type[t], der_tag(buf, sizeof(buf), tag), len); q = p + x; diff --git a/src/dprintf.c b/src/dprintf.c index 3ae1581b16e8..027a64f39e29 100644 --- a/src/dprintf.c +++ b/src/dprintf.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -28,7 +28,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: dprintf.c,v 1.1 2015/11/13 15:36:14 christos Exp $") +FILE_RCSID("@(#)$File: dprintf.c,v 1.2 2018/09/09 20:33:28 christos Exp $") #endif /* lint */ #include <assert.h> diff --git a/src/elfclass.h b/src/elfclass.h index 5360b0b55a7e..936d8dc912e0 100644 --- a/src/elfclass.h +++ b/src/elfclass.h @@ -1,7 +1,7 @@ /* * Copyright (c) Christos Zoulas 2008. * All Rights Reserved. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -11,7 +11,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -41,8 +41,8 @@ return toomany(ms, "program headers", phnum); flags |= FLAGS_IS_CORE; if (dophn_core(ms, clazz, swap, fd, - (off_t)elf_getu(swap, elfhdr.e_phoff), phnum, - (size_t)elf_getu16(swap, elfhdr.e_phentsize), + CAST(off_t, elf_getu(swap, elfhdr.e_phoff)), phnum, + CAST(size_t, elf_getu16(swap, elfhdr.e_phentsize)), fsize, &flags, ¬ecount) == -1) return -1; break; @@ -56,8 +56,8 @@ if (shnum > ms->elf_shnum_max) return toomany(ms, "section", shnum); if (dophn_exec(ms, clazz, swap, fd, - (off_t)elf_getu(swap, elfhdr.e_phoff), phnum, - (size_t)elf_getu16(swap, elfhdr.e_phentsize), + CAST(off_t, elf_getu(swap, elfhdr.e_phoff)), phnum, + CAST(size_t, elf_getu16(swap, elfhdr.e_phentsize)), fsize, shnum, &flags, ¬ecount) == -1) return -1; /*FALLTHROUGH*/ @@ -66,10 +66,10 @@ if (shnum > ms->elf_shnum_max) return toomany(ms, "section headers", shnum); if (doshn(ms, clazz, swap, fd, - (off_t)elf_getu(swap, elfhdr.e_shoff), shnum, - (size_t)elf_getu16(swap, elfhdr.e_shentsize), + CAST(off_t, elf_getu(swap, elfhdr.e_shoff)), shnum, + CAST(size_t, elf_getu16(swap, elfhdr.e_shentsize)), fsize, elf_getu16(swap, elfhdr.e_machine), - (int)elf_getu16(swap, elfhdr.e_shstrndx), + CAST(int, elf_getu16(swap, elfhdr.e_shstrndx)), &flags, ¬ecount) == -1) return -1; break; diff --git a/src/encoding.c b/src/encoding.c index 3e7b9e584b08..76244f87f95b 100644 --- a/src/encoding.c +++ b/src/encoding.c @@ -35,7 +35,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: encoding.c,v 1.14 2017/11/02 20:25:39 christos Exp $") +FILE_RCSID("@(#)$File: encoding.c,v 1.20 2019/04/15 16:48:41 christos Exp $") #endif /* lint */ #include "magic.h" @@ -49,6 +49,7 @@ private int looks_utf8_with_BOM(const unsigned char *, size_t, unichar *, size_t *); private int looks_utf7(const unsigned char *, size_t, unichar *, size_t *); private int looks_ucs16(const unsigned char *, size_t, unichar *, size_t *); +private int looks_ucs32(const unsigned char *, size_t, unichar *, size_t *); private int looks_latin1(const unsigned char *, size_t, unichar *, size_t *); private int looks_extended(const unsigned char *, size_t, unichar *, size_t *); private void from_ebcdic(const unsigned char *, size_t, unsigned char *); @@ -69,7 +70,7 @@ protected int file_encoding(struct magic_set *ms, const struct buffer *b, unichar **ubuf, size_t *ulen, const char **code, const char **code_mime, const char **type) { - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; size_t mlen; int rv = 1, ucs_type; @@ -88,12 +89,13 @@ file_encoding(struct magic_set *ms, const struct buffer *b, unichar **ubuf, *code_mime = "binary"; mlen = (nbytes + 1) * sizeof((*ubuf)[0]); - if ((*ubuf = CAST(unichar *, calloc((size_t)1, mlen))) == NULL) { + if ((*ubuf = CAST(unichar *, calloc(CAST(size_t, 1), mlen))) == NULL) { file_oomem(ms, mlen); goto done; } mlen = (nbytes + 1) * sizeof(nbuf[0]); - if ((nbuf = CAST(unsigned char *, calloc((size_t)1, mlen))) == NULL) { + if ((nbuf = CAST(unsigned char *, + calloc(CAST(size_t, 1), mlen))) == NULL) { file_oomem(ms, mlen); goto done; } @@ -116,6 +118,15 @@ file_encoding(struct magic_set *ms, const struct buffer *b, unichar **ubuf, DPRINTF(("utf8 %" SIZE_T_FORMAT "u\n", *ulen)); *code = "UTF-8 Unicode"; *code_mime = "utf-8"; + } else if ((ucs_type = looks_ucs32(buf, nbytes, *ubuf, ulen)) != 0) { + if (ucs_type == 1) { + *code = "Little-endian UTF-32 Unicode"; + *code_mime = "utf-32le"; + } else { + *code = "Big-endian UTF-32 Unicode"; + *code_mime = "utf-32be"; + } + DPRINTF(("ucs32 %" SIZE_T_FORMAT "u\n", *ulen)); } else if ((ucs_type = looks_ucs16(buf, nbytes, *ubuf, ulen)) != 0) { if (ucs_type == 1) { *code = "Little-endian UTF-16 Unicode"; @@ -410,7 +421,7 @@ looks_utf7(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) } private int -looks_ucs16(const unsigned char *buf, size_t nbytes, unichar *ubuf, +looks_ucs16(const unsigned char *bf, size_t nbytes, unichar *ubf, size_t *ulen) { int bigend; @@ -419,9 +430,9 @@ looks_ucs16(const unsigned char *buf, size_t nbytes, unichar *ubuf, if (nbytes < 2) return 0; - if (buf[0] == 0xff && buf[1] == 0xfe) + if (bf[0] == 0xff && bf[1] == 0xfe) bigend = 0; - else if (buf[0] == 0xfe && buf[1] == 0xff) + else if (bf[0] == 0xfe && bf[1] == 0xff) bigend = 1; else return 0; @@ -432,20 +443,64 @@ looks_ucs16(const unsigned char *buf, size_t nbytes, unichar *ubuf, /* XXX fix to properly handle chars > 65536 */ if (bigend) - ubuf[(*ulen)++] = buf[i + 1] + 256 * buf[i]; + ubf[(*ulen)++] = bf[i + 1] + | (CAST(unichar, bf[i]) << 8); else - ubuf[(*ulen)++] = buf[i] + 256 * buf[i + 1]; + ubf[(*ulen)++] = bf[i] + | (CAST(unichar, bf[i + 1]) << 8); - if (ubuf[*ulen - 1] == 0xfffe) + if (ubf[*ulen - 1] == 0xfffe) return 0; - if (ubuf[*ulen - 1] < 128 && - text_chars[(size_t)ubuf[*ulen - 1]] != T) + if (ubf[*ulen - 1] < 128 && + text_chars[CAST(size_t, ubf[*ulen - 1])] != T) return 0; } return 1 + bigend; } +private int +looks_ucs32(const unsigned char *bf, size_t nbytes, unichar *ubf, + size_t *ulen) +{ + int bigend; + size_t i; + + if (nbytes < 4) + return 0; + + if (bf[0] == 0xff && bf[1] == 0xfe && bf[2] == 0 && bf[3] == 0) + bigend = 0; + else if (bf[0] == 0 && bf[1] == 0 && bf[2] == 0xfe && bf[3] == 0xff) + bigend = 1; + else + return 0; + + *ulen = 0; + + for (i = 4; i + 3 < nbytes; i += 4) { + /* XXX fix to properly handle chars > 65536 */ + + if (bigend) + ubf[(*ulen)++] = CAST(unichar, bf[i + 3]) + | (CAST(unichar, bf[i + 2]) << 8) + | (CAST(unichar, bf[i + 1]) << 16) + | (CAST(unichar, bf[i]) << 24); + else + ubf[(*ulen)++] = CAST(unichar, bf[i + 0]) + | (CAST(unichar, bf[i + 1]) << 8) + | (CAST(unichar, bf[i + 2]) << 16) + | (CAST(unichar, bf[i + 3]) << 24); + + if (ubf[*ulen - 1] == 0xfffe) + return 0; + if (ubf[*ulen - 1] < 128 && + text_chars[CAST(size_t, ubf[*ulen - 1])] != T) + return 0; + } + + return 1 + bigend; +} #undef F #undef T #undef I diff --git a/src/file.c b/src/file.c index 87dd1bb60f4c..5b60b95f4240 100644 --- a/src/file.c +++ b/src/file.c @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: file.c,v 1.175 2018/03/02 16:11:37 christos Exp $") +FILE_RCSID("@(#)$File: file.c,v 1.181 2019/03/28 20:54:03 christos Exp $") #endif /* lint */ #include "magic.h" @@ -128,6 +128,7 @@ private const struct { { "encoding", MAGIC_NO_CHECK_ENCODING }, { "soft", MAGIC_NO_CHECK_SOFT }, { "tar", MAGIC_NO_CHECK_TAR }, + { "json", MAGIC_NO_CHECK_JSON }, { "text", MAGIC_NO_CHECK_TEXT }, /* synonym for ascii */ { "tokens", MAGIC_NO_CHECK_TOKENS }, /* OBSOLETE: ignored for backwards compatibility */ }; @@ -136,14 +137,15 @@ private struct { const char *name; int tag; size_t value; + int set; } pm[] = { - { "indir", MAGIC_PARAM_INDIR_MAX, 0 }, - { "name", MAGIC_PARAM_NAME_MAX, 0 }, - { "elf_phnum", MAGIC_PARAM_ELF_PHNUM_MAX, 0 }, - { "elf_shnum", MAGIC_PARAM_ELF_SHNUM_MAX, 0 }, - { "elf_notes", MAGIC_PARAM_ELF_NOTES_MAX, 0 }, - { "regex", MAGIC_PARAM_REGEX_MAX, 0 }, - { "bytes", MAGIC_PARAM_BYTES_MAX, 0 }, + { "indir", MAGIC_PARAM_INDIR_MAX, 0, 0 }, + { "name", MAGIC_PARAM_NAME_MAX, 0, 0 }, + { "elf_phnum", MAGIC_PARAM_ELF_PHNUM_MAX, 0, 0 }, + { "elf_shnum", MAGIC_PARAM_ELF_SHNUM_MAX, 0, 0 }, + { "elf_notes", MAGIC_PARAM_ELF_NOTES_MAX, 0, 0 }, + { "regex", MAGIC_PARAM_REGEX_MAX, 0, 0 }, + { "bytes", MAGIC_PARAM_BYTES_MAX, 0, 0 }, }; private int posixly; @@ -184,9 +186,7 @@ main(int argc, char *argv[]) char *progname; /* makes islower etc work for other langs */ -#ifdef HAVE_SETLOCALE (void)setlocale(LC_CTYPE, ""); -#endif #ifdef __EMX__ /* sh-like wildcard expansion! Shouldn't hurt at least ... */ @@ -242,11 +242,11 @@ main(int argc, char *argv[]) flags |= MAGIC_ERROR; break; case 'e': - for (i = 0; i < sizeof(nv) / sizeof(nv[0]); i++) + for (i = 0; i < __arraycount(nv); i++) if (strcmp(nv[i].name, optarg) == 0) break; - if (i == sizeof(nv) / sizeof(nv[0])) + if (i == __arraycount(nv)) errflg++; else flags |= nv[i].value; @@ -398,7 +398,8 @@ main(int argc, char *argv[]) } else { size_t j, wid, nw; - for (wid = 0, j = (size_t)optind; j < (size_t)argc; j++) { + for (wid = 0, j = CAST(size_t, optind); j < CAST(size_t, argc); + j++) { nw = file_mbswidth(argv[j]); if (nw > wid) wid = nw; @@ -426,7 +427,7 @@ applyparam(magic_t magic) size_t i; for (i = 0; i < __arraycount(pm); i++) { - if (pm[i].value == 0) + if (!pm[i].set) continue; if (magic_setparam(magic, pm[i].tag, &pm[i].value) == -1) file_err(EXIT_FAILURE, "Can't set %s", pm[i].name); @@ -446,6 +447,7 @@ setparam(const char *p) if (strncmp(p, pm[i].name, s - p) != 0) continue; pm[i].value = atoi(s + 1); + pm[i].set = 1; return; } badparm: @@ -534,9 +536,8 @@ process(struct magic_set *ms, const char *inname, int wid) (void)putc('\0', stdout); if (nulsep < 2) { (void)printf("%s", separator); - (void)printf("%*s ", - (int) (nopad ? 0 : (wid - file_mbswidth(inname))), - ""); + (void)printf("%*s ", CAST(int, nopad ? 0 + : (wid - file_mbswidth(inname))), ""); } } @@ -563,8 +564,8 @@ file_mbswidth(const char *s) while (n > 0) { bytesconsumed = mbrtowc(&nextchar, s, n, &state); - if (bytesconsumed == (size_t)(-1) || - bytesconsumed == (size_t)(-2)) { + if (bytesconsumed == CAST(size_t, -1) || + bytesconsumed == CAST(size_t, -2)) { /* Something went wrong, return something reasonable */ return old_n; } @@ -623,13 +624,13 @@ docprint(const char *opts, int def) for (sp = p - 1; sp > opts && *sp == ' '; sp--) continue; - fprintf(stdout, "%.*s", (int)(p - opts), opts); + fprintf(stdout, "%.*s", CAST(int, p - opts), opts); comma = 0; for (i = 0; i < __arraycount(nv); i++) { fprintf(stdout, "%s%s", comma++ ? ", " : "", nv[i].name); - if (i && i % 5 == 0) { - fprintf(stdout, ",\n%*s", (int)(p - sp - 1), ""); + if (i && i % 5 == 0 && i != __arraycount(nv) - 1) { + fprintf(stdout, ",\n%*s", CAST(int, p - sp - 1), ""); comma = 0; } } @@ -653,7 +654,7 @@ help(void) #include "file_opts.h" #undef OPT #undef OPT_LONGONLY - fprintf(stdout, "\nReport bugs to http://bugs.gw.com/\n"); + fprintf(stdout, "\nReport bugs to https://bugs.astron.com/\n"); exit(EXIT_SUCCESS); } diff --git a/src/file.h b/src/file.h index 57a84a80c0d5..69a586ab7323 100644 --- a/src/file.h +++ b/src/file.h @@ -27,7 +27,7 @@ */ /* * file.h - definitions for file(1) program - * @(#)$File: file.h,v 1.193 2018/05/24 18:09:17 christos Exp $ + * @(#)$File: file.h,v 1.206 2019/05/07 02:27:11 christos Exp $ */ #ifndef __file_h__ @@ -40,6 +40,9 @@ #ifndef __STDC_LIMIT_MACROS #define __STDC_LIMIT_MACROS #endif +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif #ifdef WIN32 #ifdef _WIN64 @@ -379,7 +382,7 @@ struct mlist { #define CCAST(T, b) const_cast<T>(b) #else #define CAST(T, b) ((T)(b)) -#define RCAST(T, b) ((T)(b)) +#define RCAST(T, b) ((T)(uintptr_t)(b)) #define CCAST(T, b) ((T)(uintptr_t)(b)) #endif @@ -450,12 +453,14 @@ struct stat; protected const char *file_fmttime(uint64_t, int, char *); protected struct magic_set *file_ms_alloc(int); protected void file_ms_free(struct magic_set *); -protected int file_buffer(struct magic_set *, int, const char *, const void *, - size_t); +protected int file_default(struct magic_set *, size_t); +protected int file_buffer(struct magic_set *, int, struct stat *, const char *, + const void *, size_t); protected int file_fsmagic(struct magic_set *, const char *, struct stat *); protected int file_pipe2file(struct magic_set *, int, const void *, size_t); protected int file_vprintf(struct magic_set *, const char *, va_list) __attribute__((__format__(__printf__, 2, 0))); +protected int file_separator(struct magic_set *); protected size_t file_printedlen(const struct magic_set *); protected int file_replace(struct magic_set *, const char *, const char *); protected int file_printf(struct magic_set *, const char *, ...) @@ -473,6 +478,7 @@ protected int file_ascmagic_with_encoding(struct magic_set *, const struct buffer *, unichar *, size_t, const char *, const char *, int); protected int file_encoding(struct magic_set *, const struct buffer *, unichar **, size_t *, const char **, const char **, const char **); +protected int file_is_json(struct magic_set *, const struct buffer *); protected int file_is_tar(struct magic_set *, const struct buffer *); protected int file_softmagic(struct magic_set *, const struct buffer *, uint16_t *, uint16_t *, int, int); @@ -501,19 +507,18 @@ protected int file_looks_utf8(const unsigned char *, size_t, unichar *, size_t *); protected size_t file_pstring_length_size(const struct magic *); protected size_t file_pstring_get_length(const struct magic *, const char *); -protected char * file_printable(char *, size_t, const char *); +protected char * file_printable(char *, size_t, const char *, size_t); #ifdef __EMX__ protected int file_os2_apptype(struct magic_set *, const char *, const void *, size_t); #endif /* __EMX__ */ -protected void buffer_init(struct buffer *, int, const void *, size_t); +protected void buffer_init(struct buffer *, int, const struct stat *, + const void *, size_t); protected void buffer_fini(struct buffer *); protected int buffer_fill(const struct buffer *); -#if defined(HAVE_LOCALE_H) #include <locale.h> -#endif #if defined(HAVE_XLOCALE_H) #include <xlocale.h> #endif @@ -550,17 +555,6 @@ extern const char *file_names[]; extern const size_t file_nnames; #endif -#ifndef HAVE_STRERROR -extern int sys_nerr; -extern char *sys_errlist[]; -#define strerror(e) \ - (((e) >= 0 && (e) < sys_nerr) ? sys_errlist[(e)] : "Unknown error") -#endif - -#ifndef HAVE_STRTOUL -#define strtoul(a, b, c) strtol(a, b, c) -#endif - #ifndef HAVE_PREAD ssize_t pread(int, void *, size_t, off_t); #endif @@ -600,17 +594,17 @@ struct tm *gmtime_r(const time_t *, struct tm *); struct tm *localtime_r(const time_t *, struct tm *); #endif #ifndef HAVE_FMTCHECK -const char *fmtcheck(const char *, const char *) +const char *fmtcheck(const char *, const char *) __attribute__((__format_arg__(2))); #endif #ifdef HAVE_LIBSECCOMP -// basic filter +// basic filter // this mode should not interfere with normal operations // only some dangerous syscalls are blacklisted int enable_sandbox_basic(void); -// enhanced filter +// enhanced filter // this mode allows only the necessary syscalls used during normal operation // extensive testing required !!! int enable_sandbox_full(void); @@ -634,6 +628,9 @@ protected void file_warnx(const char *, ...) #ifndef O_BINARY #define O_BINARY 0 #endif +#ifndef O_NONBLOCK +#define O_NONBLOCK 0 +#endif #ifndef __cplusplus #if defined(__GNUC__) && (__GNUC__ >= 3) diff --git a/src/fmtcheck.c b/src/fmtcheck.c index 486aa08681d3..fcad436b8eba 100644 --- a/src/fmtcheck.c +++ b/src/fmtcheck.c @@ -235,7 +235,7 @@ fmtcheck(const char *f1, const char *f2) EFT f1t, f2t; if (!f1) return f2; - + f1p = f1; f1t = FMTCHECK_START; f2p = f2; diff --git a/src/fsmagic.c b/src/fsmagic.c index c0a437a8f6d0..25c4f811567f 100644 --- a/src/fsmagic.c +++ b/src/fsmagic.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: fsmagic.c,v 1.77 2017/05/24 19:17:50 christos Exp $") +FILE_RCSID("@(#)$File: fsmagic.c,v 1.80 2019/04/23 18:59:27 christos Exp $") #endif /* lint */ #include "magic.h" @@ -46,11 +46,14 @@ FILE_RCSID("@(#)$File: fsmagic.c,v 1.77 2017/05/24 19:17:50 christos Exp $") # include <sys/mkdev.h> # define HAVE_MAJOR #endif -#ifdef MAJOR_IN_SYSMACROS +#ifdef HAVE_SYS_SYSMACROS_H # include <sys/sysmacros.h> +#endif +#ifdef MAJOR_IN_SYSMACROS # define HAVE_MAJOR #endif -#ifdef major /* Might be defined in sys/types.h. */ +#if defined(major) && !defined(HAVE_MAJOR) +/* Might be defined in sys/types.h. */ # define HAVE_MAJOR #endif #ifdef WIN32 @@ -77,7 +80,7 @@ bad_link(struct magic_set *ms, int err, char *buf) file_error(ms, err, "broken symbolic link to %s", buf); return -1; - } + } if (file_printf(ms, "broken symbolic link to %s", buf) == -1) return -1; } @@ -174,17 +177,17 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) return -1; #endif #ifdef S_ISGID - if (sb->st_mode & S_ISGID) + if (sb->st_mode & S_ISGID) if (file_printf(ms, "%ssetgid", COMMA) == -1) return -1; #endif #ifdef S_ISVTX - if (sb->st_mode & S_ISVTX) + if (sb->st_mode & S_ISVTX) if (file_printf(ms, "%ssticky", COMMA) == -1) return -1; #endif } - + switch (sb->st_mode & S_IFMT) { case S_IFDIR: if (mime) { @@ -196,7 +199,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) break; #ifdef S_IFCHR case S_IFCHR: - /* + /* * If -s has been specified, treat character special files * like ordinary files. Otherwise, just report that they * are block special files and go on to the next file. @@ -231,7 +234,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) #endif #ifdef S_IFBLK case S_IFBLK: - /* + /* * If -s has been specified, treat block special files * like ordinary files. Otherwise, just report that they * are block special files and go on to the next file. @@ -308,6 +311,15 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) buf[nch] = '\0'; /* readlink(2) does not do this */ /* If broken symlink, say so and quit early. */ +#ifdef __linux__ + /* + * linux procfs/devfs makes symlinks like pipe:[3515864880] + * that we can't stat their readlink output, so stat the + * original filename instead. + */ + if (stat(fn, &tstatbuf) < 0) + return bad_link(ms, errno, buf); +#else if (*buf == '/') { if (stat(buf, &tstatbuf) < 0) return bad_link(ms, errno, buf); @@ -320,7 +332,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) } else { if (tmp - fn + 1 > BUFSIZ) { if (ms->flags & MAGIC_ERROR) { - file_error(ms, 0, + file_error(ms, 0, "path too long: `%s'", buf); return -1; } @@ -345,6 +357,7 @@ file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) if (stat(tmp, &tstatbuf) < 0) return bad_link(ms, errno, buf); } +#endif /* Otherwise, handle it. */ if ((ms->flags & MAGIC_SYMLINK) != 0) { diff --git a/src/funcs.c b/src/funcs.c index 0bf92fe1c250..23e7f32e63d2 100644 --- a/src/funcs.c +++ b/src/funcs.c @@ -27,7 +27,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: funcs.c,v 1.95 2018/05/24 18:09:17 christos Exp $") +FILE_RCSID("@(#)$File: funcs.c,v 1.104 2019/05/07 02:27:11 christos Exp $") #endif /* lint */ #include "magic.h" @@ -42,9 +42,7 @@ FILE_RCSID("@(#)$File: funcs.c,v 1.95 2018/05/24 18:09:17 christos Exp $") #if defined(HAVE_WCTYPE_H) #include <wctype.h> #endif -#if defined(HAVE_LIMITS_H) #include <limits.h> -#endif #ifndef SIZE_MAX #define SIZE_MAX ((size_t)~0) @@ -107,13 +105,13 @@ file_error_core(struct magic_set *ms, int error, const char *f, va_list va, if (lineno != 0) { free(ms->o.buf); ms->o.buf = NULL; - file_printf(ms, "line %" SIZE_T_FORMAT "u:", lineno); + (void)file_printf(ms, "line %" SIZE_T_FORMAT "u:", lineno); } if (ms->o.buf && *ms->o.buf) - file_printf(ms, " "); - file_vprintf(ms, f, va); + (void)file_printf(ms, " "); + (void)file_vprintf(ms, f, va); if (error > 0) - file_printf(ms, " (%s)", strerror(error)); + (void)file_printf(ms, " (%s)", strerror(error)); ms->event_flags |= EVENT_HAD_ERR; ms->error = error; } @@ -162,36 +160,70 @@ file_badread(struct magic_set *ms) #ifndef COMPILE_ONLY +protected int +file_separator(struct magic_set *ms) +{ + return file_printf(ms, "\n- "); +} + static int checkdone(struct magic_set *ms, int *rv) { if ((ms->flags & MAGIC_CONTINUE) == 0) return 1; - if (file_printf(ms, "\n- ") == -1) + if (file_separator(ms) == -1) *rv = -1; return 0; } +protected int +file_default(struct magic_set *ms, size_t nb) +{ + if (ms->flags & MAGIC_MIME) { + if ((ms->flags & MAGIC_MIME_TYPE) && + file_printf(ms, "application/%s", + nb ? "octet-stream" : "x-empty") == -1) + return -1; + return 1; + } + if (ms->flags & MAGIC_APPLE) { + if (file_printf(ms, "UNKNUNKN") == -1) + return -1; + return 1; + } + if (ms->flags & MAGIC_EXTENSION) { + if (file_printf(ms, "???") == -1) + return -1; + return 1; + } + return 0; +} + +/* + * The magic detection functions return: + * 1: found + * 0: not found + * -1: error + */ /*ARGSUSED*/ protected int -file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((__unused__)), +file_buffer(struct magic_set *ms, int fd, struct stat *st, + const char *inname __attribute__ ((__unused__)), const void *buf, size_t nb) { int m = 0, rv = 0, looks_text = 0; const char *code = NULL; const char *code_mime = "binary"; - const char *type = "application/octet-stream"; const char *def = "data"; const char *ftype = NULL; char *rbuf = NULL; struct buffer b; - buffer_init(&b, fd, buf, nb); + buffer_init(&b, fd, st, buf, nb); ms->mode = b.st.st_mode; if (nb == 0) { def = "empty"; - type = "application/x-empty"; goto simple; } else if (nb == 1) { def = "very short file (no magic)"; @@ -240,6 +272,17 @@ file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((__u } } + /* Check if we have a JSON file */ + if ((ms->flags & MAGIC_NO_CHECK_JSON) == 0) { + m = file_is_json(ms, &b); + if ((ms->flags & MAGIC_DEBUG) != 0) + (void)fprintf(stderr, "[try json %d]\n", m); + if (m) { + if (checkdone(ms, &rv)) + goto done; + } + } + /* Check if we have a CDF file */ if ((ms->flags & MAGIC_NO_CHECK_CDF) == 0) { m = file_trycdf(ms, &b); @@ -268,7 +311,7 @@ file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((__u rv = file_tryelf(ms, &b); rbuf = file_pop_buffer(ms, pb); - if (rv != 1) { + if (rv == -1) { free(rbuf); rbuf = NULL; } @@ -299,27 +342,18 @@ file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((__u if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "[try ascmagic %d]\n", m); if (m) { - if (checkdone(ms, &rv)) - goto done; + goto done; } } simple: /* give up */ - m = 1; - if (ms->flags & MAGIC_MIME) { - if ((ms->flags & MAGIC_MIME_TYPE) && - file_printf(ms, "%s", type) == -1) - rv = -1; - } else if (ms->flags & MAGIC_APPLE) { - if (file_printf(ms, "UNKNUNKN") == -1) - rv = -1; - } else if (ms->flags & MAGIC_EXTENSION) { - if (file_printf(ms, "???") == -1) - rv = -1; - } else { - if (file_printf(ms, "%s", def) == -1) - rv = -1; + if (m == 0) { + m = 1; + rv = file_default(ms, nb); + if (rv == 0) + if (file_printf(ms, "%s", def) == -1) + rv = -1; } done: if ((ms->flags & MAGIC_MIME_ENCODING) != 0) { @@ -364,9 +398,9 @@ file_reset(struct magic_set *ms, int checkloaded) #define OCTALIFY(n, o) \ /*LINTED*/ \ (void)(*(n)++ = '\\', \ - *(n)++ = (((uint32_t)*(o) >> 6) & 3) + '0', \ - *(n)++ = (((uint32_t)*(o) >> 3) & 7) + '0', \ - *(n)++ = (((uint32_t)*(o) >> 0) & 7) + '0', \ + *(n)++ = ((CAST(uint32_t, *(o)) >> 6) & 3) + '0', \ + *(n)++ = ((CAST(uint32_t, *(o)) >> 3) & 7) + '0', \ + *(n)++ = ((CAST(uint32_t, *(o)) >> 0) & 7) + '0', \ (o)++) protected const char * @@ -412,9 +446,9 @@ file_getbuffer(struct magic_set *ms) while (op < eop) { bytesconsumed = mbrtowc(&nextchar, op, - (size_t)(eop - op), &state); - if (bytesconsumed == (size_t)(-1) || - bytesconsumed == (size_t)(-2)) { + CAST(size_t, eop - op), &state); + if (bytesconsumed == CAST(size_t, -1) || + bytesconsumed == CAST(size_t, -2)) { mb_conv = 0; break; } @@ -437,7 +471,7 @@ file_getbuffer(struct magic_set *ms) #endif for (np = ms->o.pbuf, op = ms->o.buf; *op;) { - if (isprint((unsigned char)*op)) { + if (isprint(CAST(unsigned char, *op))) { *np++ = *op++; } else { OCTALIFY(np, op); @@ -595,12 +629,13 @@ file_pop_buffer(struct magic_set *ms, file_pushbuf_t *pb) * convert string to ascii printable format. */ protected char * -file_printable(char *buf, size_t bufsiz, const char *str) +file_printable(char *buf, size_t bufsiz, const char *str, size_t slen) { - char *ptr, *eptr; - const unsigned char *s = (const unsigned char *)str; + char *ptr, *eptr = buf + bufsiz - 1; + const unsigned char *s = RCAST(const unsigned char *, str); + const unsigned char *es = s + slen; - for (ptr = buf, eptr = ptr + bufsiz - 1; ptr < eptr && *s; s++) { + for (ptr = buf; ptr < eptr && s < es && *s; s++) { if (isprint(*s)) { *ptr++ = *s; continue; diff --git a/src/getopt_long.c b/src/getopt_long.c index d357dc30f444..43c4245619ef 100644 --- a/src/getopt_long.c +++ b/src/getopt_long.c @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: getopt_long.c,v 1.6 2009/02/13 18:48:05 christos Exp $") +FILE_RCSID("@(#)$File: getopt_long.c,v 1.7 2018/09/09 20:33:28 christos Exp $") #endif /* lint */ #include <assert.h> @@ -124,7 +124,7 @@ gcd(a, b) b = c; c = a % b; } - + return b; } @@ -223,7 +223,7 @@ start: place = EMSG; if (IN_ORDER) { /* - * GNU extension: + * GNU extension: * return non-option as argument to option 1 */ optarg = nargv[optind++]; @@ -269,7 +269,7 @@ start: } if (optchar == 'W' && oli[1] == ';') { /* -W long-option */ /* XXX: what if no long options provided (called by getopt)? */ - if (*place) + if (*place) return -2; if (++optind >= nargc) { /* no arg */ @@ -404,7 +404,7 @@ getopt_long(nargc, nargv, options, long_options, idx) has_equal++; } else current_argv_len = strlen(current_argv); - + for (i = 0; long_options[i].name; i++) { /* find matching long option */ if (strncmp(current_argv, long_options[i].name, @@ -488,7 +488,7 @@ getopt_long(nargc, nargv, options, long_options, idx) if (long_options[match].flag) { *long_options[match].flag = long_options[match].val; retval = 0; - } else + } else retval = long_options[match].val; if (idx) *idx = match; diff --git a/src/is_json.c b/src/is_json.c new file mode 100644 index 000000000000..206ec3795e54 --- /dev/null +++ b/src/is_json.c @@ -0,0 +1,462 @@ +/*- + * Copyright (c) 2018 Christos Zoulas + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS + * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * Parse JSON object serialization format (RFC-7159) + */ + +#ifndef TEST +#include "file.h" + +#ifndef lint +FILE_RCSID("@(#)$File: is_json.c,v 1.13 2019/03/02 01:08:10 christos Exp $") +#endif + +#include <string.h> +#include "magic.h" +#endif + +#ifdef DEBUG +#include <stdio.h> +#define DPRINTF(a, b, c) \ + printf("%s [%.2x/%c] %.20s\n", (a), *(b), *(b), (const char *)(c)) +#else +#define DPRINTF(a, b, c) do { } while (/*CONSTCOND*/0) +#endif + +#define JSON_ARRAY 0 +#define JSON_CONSTANT 1 +#define JSON_NUMBER 2 +#define JSON_OBJECT 3 +#define JSON_STRING 4 +#define JSON_ARRAYN 5 +#define JSON_MAX 6 + +/* + * if JSON_COUNT != 0: + * count all the objects, require that we have the whole data file + * otherwise: + * stop if we find an object or an array + */ +#ifndef JSON_COUNT +#define JSON_COUNT 0 +#endif + +static int json_parse(const unsigned char **, const unsigned char *, size_t *, + size_t); + +static int +json_isspace(const unsigned char uc) +{ + switch (uc) { + case ' ': + case '\n': + case '\r': + case '\t': + return 1; + default: + return 0; + } +} + +static int +json_isdigit(unsigned char uc) +{ + switch (uc) { + case '0': case '1': case '2': case '3': case '4': + case '5': case '6': case '7': case '8': case '9': + return 1; + default: + return 0; + } +} + +static int +json_isxdigit(unsigned char uc) +{ + if (json_isdigit(uc)) + return 1; + switch (uc) { + case 'a': case 'b': case 'c': case 'd': case 'e': case 'f': + case 'A': case 'B': case 'C': case 'D': case 'E': case 'F': + return 1; + default: + return 0; + } +} + +static const unsigned char * +json_skip_space(const unsigned char *uc, const unsigned char *ue) +{ + while (uc < ue && json_isspace(*uc)) + uc++; + return uc; +} + +static int +json_parse_string(const unsigned char **ucp, const unsigned char *ue) +{ + const unsigned char *uc = *ucp; + size_t i; + + DPRINTF("Parse string: ", uc, *ucp); + while (uc < ue) { + switch (*uc++) { + case '\0': + goto out; + case '\\': + if (uc == ue) + goto out; + switch (*uc++) { + case '\0': + goto out; + case '"': + case '\\': + case '/': + case 'b': + case 'f': + case 'n': + case 'r': + case 't': + continue; + case 'u': + if (ue - uc < 4) { + uc = ue; + goto out; + } + for (i = 0; i < 4; i++) + if (!json_isxdigit(*uc++)) + goto out; + continue; + default: + goto out; + } + case '"': + *ucp = uc; + return 1; + default: + continue; + } + } +out: + DPRINTF("Bad string: ", uc, *ucp); + *ucp = uc; + return 0; +} + +static int +json_parse_array(const unsigned char **ucp, const unsigned char *ue, + size_t *st, size_t lvl) +{ + const unsigned char *uc = *ucp; + int more = 0; /* Array has more than 1 element */ + + DPRINTF("Parse array: ", uc, *ucp); + while (uc < ue) { + if (!json_parse(&uc, ue, st, lvl + 1)) + goto out; + if (uc == ue) + goto out; + switch (*uc) { + case ',': + more++; + uc++; + continue; + case ']': + if (more) + st[JSON_ARRAYN]++; + *ucp = uc + 1; + return 1; + default: + goto out; + } + } +out: + DPRINTF("Bad array: ", uc, *ucp); + *ucp = uc; + return 0; +} + +static int +json_parse_object(const unsigned char **ucp, const unsigned char *ue, + size_t *st, size_t lvl) +{ + const unsigned char *uc = *ucp; + DPRINTF("Parse object: ", uc, *ucp); + while (uc < ue) { + uc = json_skip_space(uc, ue); + if (uc == ue) + goto out; + if (*uc++ != '"') { + DPRINTF("not string", uc, *ucp); + goto out; + } + DPRINTF("next field", uc, *ucp); + if (!json_parse_string(&uc, ue)) { + DPRINTF("not string", uc, *ucp); + goto out; + } + uc = json_skip_space(uc, ue); + if (uc == ue) + goto out; + if (*uc++ != ':') { + DPRINTF("not colon", uc, *ucp); + goto out; + } + if (!json_parse(&uc, ue, st, lvl + 1)) { + DPRINTF("not json", uc, *ucp); + goto out; + } + if (uc == ue) + goto out; + switch (*uc++) { + case ',': + continue; + case '}': /* { */ + *ucp = uc; + DPRINTF("Good object: ", uc, *ucp); + return 1; + default: + *ucp = uc - 1; + DPRINTF("not more", uc, *ucp); + goto out; + } + } +out: + DPRINTF("Bad object: ", uc, *ucp); + *ucp = uc; + return 0; +} + +static int +json_parse_number(const unsigned char **ucp, const unsigned char *ue) +{ + const unsigned char *uc = *ucp; + int got = 0; + + DPRINTF("Parse number: ", uc, *ucp); + if (uc == ue) + return 0; + if (*uc == '-') + uc++; + + for (; uc < ue; uc++) { + if (!json_isdigit(*uc)) + break; + got = 1; + } + if (uc == ue) + goto out; + if (*uc == '.') + uc++; + for (; uc < ue; uc++) { + if (!json_isdigit(*uc)) + break; + got = 1; + } + if (uc == ue) + goto out; + if (got && (*uc == 'e' || *uc == 'E')) { + uc++; + got = 0; + if (uc == ue) + goto out; + if (*uc == '+' || *uc == '-') + uc++; + for (; uc < ue; uc++) { + if (!json_isdigit(*uc)) + break; + got = 1; + } + } +out: + if (!got) + DPRINTF("Bad number: ", uc, *ucp); + else + DPRINTF("Good number: ", uc, *ucp); + *ucp = uc; + return got; +} + +static int +json_parse_const(const unsigned char **ucp, const unsigned char *ue, + const char *str, size_t len) +{ + const unsigned char *uc = *ucp; + + DPRINTF("Parse const: ", uc, *ucp); + for (len--; uc < ue && --len;) { + if (*uc++ == *++str) + continue; + } + if (len) + DPRINTF("Bad const: ", uc, *ucp); + *ucp = uc; + return len == 0; +} + +static int +json_parse(const unsigned char **ucp, const unsigned char *ue, + size_t *st, size_t lvl) +{ + const unsigned char *uc; + int rv = 0; + int t; + + uc = json_skip_space(*ucp, ue); + if (uc == ue) + goto out; + + // Avoid recursion + if (lvl > 20) + return 0; +#if JSON_COUNT + /* bail quickly if not counting */ + if (lvl > 1 && (st[JSON_OBJECT] || st[JSON_ARRAYN])) + return 1; +#endif + + DPRINTF("Parse general: ", uc, *ucp); + switch (*uc++) { + case '"': + rv = json_parse_string(&uc, ue); + t = JSON_STRING; + break; + case '[': + rv = json_parse_array(&uc, ue, st, lvl + 1); + t = JSON_ARRAY; + break; + case '{': /* '}' */ + rv = json_parse_object(&uc, ue, st, lvl + 1); + t = JSON_OBJECT; + break; + case 't': + rv = json_parse_const(&uc, ue, "true", sizeof("true")); + t = JSON_CONSTANT; + break; + case 'f': + rv = json_parse_const(&uc, ue, "false", sizeof("false")); + t = JSON_CONSTANT; + break; + case 'n': + rv = json_parse_const(&uc, ue, "null", sizeof("null")); + t = JSON_CONSTANT; + break; + default: + --uc; + rv = json_parse_number(&uc, ue); + t = JSON_NUMBER; + break; + } + if (rv) + st[t]++; + uc = json_skip_space(uc, ue); +out: + *ucp = uc; + DPRINTF("End general: ", uc, *ucp); + if (lvl == 0) + return rv && (st[JSON_ARRAYN] || st[JSON_OBJECT]); + return rv; +} + +#ifndef TEST +int +file_is_json(struct magic_set *ms, const struct buffer *b) +{ + const unsigned char *uc = CAST(const unsigned char *, b->fbuf); + const unsigned char *ue = uc + b->flen; + size_t st[JSON_MAX]; + int mime = ms->flags & MAGIC_MIME; + + + if ((ms->flags & (MAGIC_APPLE|MAGIC_EXTENSION)) != 0) + return 0; + + memset(st, 0, sizeof(st)); + + if (!json_parse(&uc, ue, st, 0)) + return 0; + + if (mime == MAGIC_MIME_ENCODING) + return 1; + if (mime) { + if (file_printf(ms, "application/json") == -1) + return -1; + return 1; + } + if (file_printf(ms, "JSON data") == -1) + return -1; +#if JSON_COUNT +#define P(n) st[n], st[n] > 1 ? "s" : "" + if (file_printf(ms, " (%" SIZE_T_FORMAT "u object%s, %" SIZE_T_FORMAT + "u array%s, %" SIZE_T_FORMAT "u string%s, %" SIZE_T_FORMAT + "u constant%s, %" SIZE_T_FORMAT "u number%s, %" SIZE_T_FORMAT + "u >1array%s)", + P(JSON_OBJECT), P(JSON_ARRAY), P(JSON_STRING), P(JSON_CONSTANT), + P(JSON_NUMBER), P(JSON_ARRAYN)) + == -1) + return -1; +#endif + return 1; +} + +#else + +#include <sys/types.h> +#include <sys/stat.h> +#include <stdio.h> +#include <fcntl.h> +#include <unistd.h> +#include <stdlib.h> +#include <stdint.h> +#include <err.h> + +int +main(int argc, char *argv[]) +{ + int fd, rv; + struct stat st; + unsigned char *p; + size_t stats[JSON_MAX]; + + if ((fd = open(argv[1], O_RDONLY)) == -1) + err(EXIT_FAILURE, "Can't open `%s'", argv[1]); + + if (fstat(fd, &st) == -1) + err(EXIT_FAILURE, "Can't stat `%s'", argv[1]); + + if ((p = malloc(st.st_size)) == NULL) + err(EXIT_FAILURE, "Can't allocate %jd bytes", + (intmax_t)st.st_size); + if (read(fd, p, st.st_size) != st.st_size) + err(EXIT_FAILURE, "Can't read %jd bytes", + (intmax_t)st.st_size); + memset(stats, 0, sizeof(stats)); + printf("is json %d\n", json_parse((const unsigned char **)&p, + p + st.st_size, stats, 0)); + return 0; +} +#endif diff --git a/src/is_tar.c b/src/is_tar.c index 7110604f9e81..82b08051fbdd 100644 --- a/src/is_tar.c +++ b/src/is_tar.c @@ -40,7 +40,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: is_tar.c,v 1.41 2017/11/02 20:25:39 christos Exp $") +FILE_RCSID("@(#)$File: is_tar.c,v 1.44 2019/02/20 02:35:27 christos Exp $") #endif #include "magic.h" @@ -62,7 +62,7 @@ static const char tartype[][32] = { /* should be equal to messages */ protected int file_is_tar(struct magic_set *ms, const struct buffer *b) { - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; /* * Do the tar test first, because if the first file in the tar @@ -78,9 +78,13 @@ file_is_tar(struct magic_set *ms, const struct buffer *b) if (tar < 1 || tar > 3) return 0; + if (mime == MAGIC_MIME_ENCODING) + return 1; + if (file_printf(ms, "%s", mime ? "application/x-tar" : tartype[tar - 1]) == -1) return -1; + return 1; } @@ -94,7 +98,8 @@ file_is_tar(struct magic_set *ms, const struct buffer *b) private int is_tar(const unsigned char *buf, size_t nbytes) { - const union record *header = (const union record *)(const void *)buf; + const union record *header = RCAST(const union record *, + RCAST(const void *, buf)); size_t i; int sum, recsum; const unsigned char *p, *ep; @@ -143,7 +148,7 @@ from_oct(const char *where, size_t digs) if (digs == 0) return -1; - while (isspace((unsigned char)*where)) { /* Skip spaces */ + while (isspace(CAST(unsigned char, *where))) { /* Skip spaces */ where++; if (digs-- == 0) return -1; /* All blank field */ @@ -154,7 +159,7 @@ from_oct(const char *where, size_t digs) digs--; } - if (digs > 0 && *where && !isspace((unsigned char)*where)) + if (digs > 0 && *where && !isspace(CAST(unsigned char, *where))) return -1; /* Ended on non-(space/NUL) */ return value; diff --git a/src/magic.c b/src/magic.c index 1448a69bcad5..da5baf108e5d 100644 --- a/src/magic.c +++ b/src/magic.c @@ -33,7 +33,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: magic.c,v 1.102 2017/08/28 13:39:18 christos Exp $") +FILE_RCSID("@(#)$File: magic.c,v 1.111 2019/05/07 02:27:11 christos Exp $") #endif /* lint */ #include "magic.h" @@ -44,9 +44,7 @@ FILE_RCSID("@(#)$File: magic.c,v 1.102 2017/08/28 13:39:18 christos Exp $") #ifdef QUICK #include <sys/mman.h> #endif -#ifdef HAVE_LIMITS_H #include <limits.h> /* for PIPE_BUF */ -#endif #if defined(HAVE_UTIMES) # include <sys/time.h> @@ -314,7 +312,8 @@ magic_load_buffers(struct magic_set *ms, void **bufs, size_t *sizes, { if (ms == NULL) return -1; - return buffer_apprentice(ms, (struct magic **)bufs, sizes, nbufs); + return buffer_apprentice(ms, RCAST(struct magic **, bufs), + sizes, nbufs); } #endif @@ -407,7 +406,8 @@ file_or_fd(struct magic_set *ms, const char *inname, int fd) struct stat sb; ssize_t nbytes = 0; /* number of bytes read from a datafile */ int ispipe = 0; - off_t pos = (off_t)-1; + int okstat = 0; + off_t pos = CAST(off_t, -1); if (file_reset(ms, 1) == -1) goto out; @@ -435,25 +435,13 @@ file_or_fd(struct magic_set *ms, const char *inname, int fd) if (fd == STDIN_FILENO) _setmode(STDIN_FILENO, O_BINARY); #endif - - if (inname == NULL) { - if (fstat(fd, &sb) == 0 && S_ISFIFO(sb.st_mode)) - ispipe = 1; - else - pos = lseek(fd, (off_t)0, SEEK_CUR); - } else { - int flags = O_RDONLY|O_BINARY; - int okstat = stat(inname, &sb) == 0; - - if (okstat && S_ISFIFO(sb.st_mode)) { -#ifdef O_NONBLOCK - flags |= O_NONBLOCK; -#endif - ispipe = 1; - } - + if (inname != NULL) { + int flags = O_RDONLY|O_BINARY|O_NONBLOCK; errno = 0; if ((fd = open(inname, flags)) < 0) { + okstat = stat(inname, &sb) == 0; + if (okstat && S_ISFIFO(sb.st_mode)) + ispipe = 1; #ifdef WIN32 /* * Can't stat, can't open. It may have been opened in @@ -472,24 +460,29 @@ file_or_fd(struct magic_set *ms, const char *inname, int fd) rv = 0; goto done; } -#ifdef O_NONBLOCK - if ((flags = fcntl(fd, F_GETFL)) != -1) { - flags &= ~O_NONBLOCK; - (void)fcntl(fd, F_SETFL, flags); - } -#endif + } + + if (fd != -1) { + if (!okstat) + okstat = fstat(fd, &sb) == 0; + if (okstat && S_ISFIFO(sb.st_mode)) + ispipe = 1; + if (inname == NULL) + pos = lseek(fd, CAST(off_t, 0), SEEK_CUR); } /* * try looking at the first ms->bytes_max bytes */ if (ispipe) { - ssize_t r = 0; + if (fd != -1) { + ssize_t r = 0; - while ((r = sread(fd, (void *)&buf[nbytes], - (size_t)(ms->bytes_max - nbytes), 1)) > 0) { - nbytes += r; - if (r < PIPE_BUF) break; + while ((r = sread(fd, RCAST(void *, &buf[nbytes]), + CAST(size_t, ms->bytes_max - nbytes), 1)) > 0) { + nbytes += r; + if (r < PIPE_BUF) break; + } } if (nbytes == 0 && inname) { @@ -500,14 +493,14 @@ file_or_fd(struct magic_set *ms, const char *inname, int fd) goto done; } - } else { + } else if (fd != -1) { /* Windows refuses to read from a big console buffer. */ size_t howmany = #if defined(WIN32) - _isatty(fd) ? 8 * 1024 : + _isatty(fd) ? 8 * 1024 : #endif - ms->bytes_max; - if ((nbytes = read(fd, (char *)buf, howmany)) == -1) { + ms->bytes_max; + if ((nbytes = read(fd, RCAST(void *, buf), howmany)) == -1) { if (inname == NULL && fd != STDIN_FILENO) file_error(ms, errno, "cannot read fd %d", fd); else @@ -518,13 +511,13 @@ file_or_fd(struct magic_set *ms, const char *inname, int fd) } (void)memset(buf + nbytes, 0, SLOP); /* NUL terminate */ - if (file_buffer(ms, fd, inname, buf, (size_t)nbytes) == -1) + if (file_buffer(ms, fd, okstat ? &sb : NULL, inname, buf, CAST(size_t, nbytes)) == -1) goto done; rv = 0; done: free(buf); if (fd != -1) { - if (pos != (off_t)-1) + if (pos != CAST(off_t, -1)) (void)lseek(fd, pos, SEEK_SET); close_and_restore(ms, inname, fd, &sb); } @@ -544,7 +537,7 @@ magic_buffer(struct magic_set *ms, const void *buf, size_t nb) * The main work is done here! * We have the file name and/or the data buffer to be identified. */ - if (file_buffer(ms, -1, NULL, buf, nb) == -1) { + if (file_buffer(ms, -1, NULL, NULL, buf, nb) == -1) { return NULL; } return file_getbuffer(ms); @@ -598,27 +591,29 @@ magic_version(void) public int magic_setparam(struct magic_set *ms, int param, const void *val) { + if (ms == NULL) + return -1; switch (param) { case MAGIC_PARAM_INDIR_MAX: - ms->indir_max = (uint16_t)*(const size_t *)val; + ms->indir_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_NAME_MAX: - ms->name_max = (uint16_t)*(const size_t *)val; + ms->name_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_ELF_PHNUM_MAX: - ms->elf_phnum_max = (uint16_t)*(const size_t *)val; + ms->elf_phnum_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_ELF_SHNUM_MAX: - ms->elf_shnum_max = (uint16_t)*(const size_t *)val; + ms->elf_shnum_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_ELF_NOTES_MAX: - ms->elf_notes_max = (uint16_t)*(const size_t *)val; + ms->elf_notes_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_REGEX_MAX: - ms->elf_notes_max = (uint16_t)*(const size_t *)val; + ms->regex_max = CAST(uint16_t, *CAST(const size_t *, val)); return 0; case MAGIC_PARAM_BYTES_MAX: - ms->bytes_max = *(const size_t *)val; + ms->bytes_max = *CAST(const size_t *, val); return 0; default: errno = EINVAL; @@ -629,27 +624,29 @@ magic_setparam(struct magic_set *ms, int param, const void *val) public int magic_getparam(struct magic_set *ms, int param, void *val) { + if (ms == NULL) + return -1; switch (param) { case MAGIC_PARAM_INDIR_MAX: - *(size_t *)val = ms->indir_max; + *CAST(size_t *, val) = ms->indir_max; return 0; case MAGIC_PARAM_NAME_MAX: - *(size_t *)val = ms->name_max; + *CAST(size_t *, val) = ms->name_max; return 0; case MAGIC_PARAM_ELF_PHNUM_MAX: - *(size_t *)val = ms->elf_phnum_max; + *CAST(size_t *, val) = ms->elf_phnum_max; return 0; case MAGIC_PARAM_ELF_SHNUM_MAX: - *(size_t *)val = ms->elf_shnum_max; + *CAST(size_t *, val) = ms->elf_shnum_max; return 0; case MAGIC_PARAM_ELF_NOTES_MAX: - *(size_t *)val = ms->elf_notes_max; + *CAST(size_t *, val) = ms->elf_notes_max; return 0; case MAGIC_PARAM_REGEX_MAX: - *(size_t *)val = ms->regex_max; + *CAST(size_t *, val) = ms->regex_max; return 0; case MAGIC_PARAM_BYTES_MAX: - *(size_t *)val = ms->bytes_max; + *CAST(size_t *, val) = ms->bytes_max; return 0; default: errno = EINVAL; diff --git a/src/magic.h.in b/src/magic.h.in index 1134bdc886d4..0580fa21ccea 100644 --- a/src/magic.h.in +++ b/src/magic.h.in @@ -58,6 +58,7 @@ #define MAGIC_NO_CHECK_CDF 0x0040000 /* Don't check for cdf files */ #define MAGIC_NO_CHECK_TOKENS 0x0100000 /* Don't check tokens */ #define MAGIC_NO_CHECK_ENCODING 0x0200000 /* Don't check text encodings */ +#define MAGIC_NO_CHECK_JSON 0x0400000 /* Don't check for JSON files */ /* No built-in tests; only consult the magic file */ #define MAGIC_NO_CHECK_BUILTIN ( \ @@ -70,6 +71,7 @@ MAGIC_NO_CHECK_CDF | \ MAGIC_NO_CHECK_TOKENS | \ MAGIC_NO_CHECK_ENCODING | \ + MAGIC_NO_CHECK_JSON | \ 0 \ ) @@ -96,7 +98,7 @@ b\22no_check_cdf\0\ b\23no_check_reserved0\0\ b\24no_check_tokens\0\ b\25no_check_encoding\0\ -b\26no_check_reserved1\0\ +b\26no_check_json\0\ b\27no_check_reserved2\0\ b\30extension\0\ b\31transp_compression\0\ diff --git a/src/mygetopt.h b/src/mygetopt.h index ef87525eaf5b..d766762631d1 100644 --- a/src/mygetopt.h +++ b/src/mygetopt.h @@ -64,5 +64,5 @@ struct option { int getopt_long(int, char * const *, const char *, const struct option *, int *); - + #endif /* !_GETOPT_H_ */ diff --git a/src/print.c b/src/print.c index 0b918636bc5e..391a7fbe1562 100644 --- a/src/print.c +++ b/src/print.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: print.c,v 1.82 2017/02/10 18:14:01 christos Exp $") +FILE_RCSID("@(#)$File: print.c,v 1.85 2019/03/12 20:43:05 christos Exp $") #endif /* lint */ #include <string.h> @@ -43,8 +43,6 @@ FILE_RCSID("@(#)$File: print.c,v 1.82 2017/02/10 18:14:01 christos Exp $") #endif #include <time.h> -#define SZOF(a) (sizeof(a) / sizeof(a[0])) - #include "cdf.h" #ifndef COMPILE_ONLY @@ -65,9 +63,9 @@ file_mdump(struct magic *m) if (m->in_op & FILE_OPINVERSE) (void) fputc('~', stderr); (void) fprintf(stderr, "%c%u),", - ((size_t)(m->in_op & FILE_OPS_MASK) < - SZOF(optyp)) ? optyp[m->in_op & FILE_OPS_MASK] : '?', - m->in_offset); + (CAST(size_t, m->in_op & FILE_OPS_MASK) < + __arraycount(optyp)) ? + optyp[m->in_op & FILE_OPS_MASK] : '?', m->in_offset); } (void) fprintf(stderr, " %s%s", (m->flag & UNSIGNED) ? "u" : "", /* Note: type is unsigned */ @@ -78,16 +76,16 @@ file_mdump(struct magic *m) if (IS_STRING(m->type)) { if (m->str_flags) { (void) fputc('/', stderr); - if (m->str_flags & STRING_COMPACT_WHITESPACE) + if (m->str_flags & STRING_COMPACT_WHITESPACE) (void) fputc(CHAR_COMPACT_WHITESPACE, stderr); - if (m->str_flags & STRING_COMPACT_OPTIONAL_WHITESPACE) + if (m->str_flags & STRING_COMPACT_OPTIONAL_WHITESPACE) (void) fputc(CHAR_COMPACT_OPTIONAL_WHITESPACE, stderr); - if (m->str_flags & STRING_IGNORE_LOWERCASE) + if (m->str_flags & STRING_IGNORE_LOWERCASE) (void) fputc(CHAR_IGNORE_LOWERCASE, stderr); - if (m->str_flags & STRING_IGNORE_UPPERCASE) + if (m->str_flags & STRING_IGNORE_UPPERCASE) (void) fputc(CHAR_IGNORE_UPPERCASE, stderr); - if (m->str_flags & REGEX_OFFSET_START) + if (m->str_flags & REGEX_OFFSET_START) (void) fputc(CHAR_REGEX_OFFSET_START, stderr); if (m->str_flags & STRING_TEXTTEST) (void) fputc(CHAR_TEXTTEST, stderr); @@ -112,14 +110,15 @@ file_mdump(struct magic *m) (void) fprintf(stderr, "/%u", m->str_range); } else { - if ((size_t)(m->mask_op & FILE_OPS_MASK) < SZOF(optyp)) + if (CAST(size_t, m->mask_op & FILE_OPS_MASK) < + __arraycount(optyp)) (void) fputc(optyp[m->mask_op & FILE_OPS_MASK], stderr); else (void) fputc('?', stderr); - + if (m->num_mask) { (void) fprintf(stderr, "%.8llx", - (unsigned long long)m->num_mask); + CAST(unsigned long long, m->num_mask)); } } (void) fprintf(stderr, ",%c", m->reln); @@ -141,7 +140,7 @@ file_mdump(struct magic *m) case FILE_LEQUAD: case FILE_QUAD: (void) fprintf(stderr, "%" INT64_T_FORMAT "d", - (unsigned long long)m->value.q); + CAST(long long, m->value.q)); break; case FILE_PSTRING: case FILE_STRING: @@ -149,7 +148,8 @@ file_mdump(struct magic *m) case FILE_BESTRING16: case FILE_LESTRING16: case FILE_SEARCH: - file_showstr(stderr, m->value.s, (size_t)m->vallen); + file_showstr(stderr, m->value.s, + CAST(size_t, m->vallen)); break; case FILE_DATE: case FILE_LEDATE: @@ -217,11 +217,11 @@ file_magwarn(struct magic_set *ms, const char *f, ...) va_list va; /* cuz we use stdout for most, stderr here */ - (void) fflush(stdout); + (void) fflush(stdout); if (ms->file) (void) fprintf(stderr, "%s, %lu: ", ms->file, - (unsigned long)ms->line); + CAST(unsigned long, ms->line)); (void) fprintf(stderr, "Warning: "); va_start(va, f); (void) vfprintf(stderr, f, va); @@ -243,7 +243,7 @@ file_fmttime(uint64_t v, int flags, char *buf) } else { // XXX: perhaps detect and print something if overflow // on 32 bit time_t? - t = (time_t)v; + t = CAST(time_t, v); } if (flags & FILE_T_LOCAL) { diff --git a/src/readcdf.c b/src/readcdf.c index 4b86e6fed08d..e6ea8e47fb6b 100644 --- a/src/readcdf.c +++ b/src/readcdf.c @@ -26,7 +26,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: readcdf.c,v 1.67 2018/04/15 19:57:07 christos Exp $") +FILE_RCSID("@(#)$File: readcdf.c,v 1.73 2019/03/12 20:43:05 christos Exp $") #endif #include <assert.h> @@ -39,10 +39,6 @@ FILE_RCSID("@(#)$File: readcdf.c,v 1.67 2018/04/15 19:57:07 christos Exp $") #include "cdf.h" #include "magic.h" -#ifndef __arraycount -#define __arraycount(a) (sizeof(a) / sizeof(a[0])) -#endif - #define NOTMIME(ms) (((ms)->flags & MAGIC_MIME) == 0) static const struct nv { @@ -204,7 +200,7 @@ cdf_file_property_info(struct magic_set *ms, const cdf_property_info_t *info, && len--; s += k) { if (*s == '\0') break; - if (isprint((unsigned char)*s)) + if (isprint(CAST(unsigned char, *s))) vbuf[j++] = *s; } if (j == sizeof(vbuf)) @@ -252,7 +248,7 @@ cdf_file_property_info(struct magic_set *ms, const cdf_property_info_t *info, return -1; } } - if (!NOTMIME(ms)) { + if (ms->flags & MAGIC_MIME_TYPE) { if (str == NULL) return 0; if (file_printf(ms, "application/%s", str) == -1) @@ -285,7 +281,7 @@ cdf_file_catalog(struct magic_set *ms, const cdf_header_t *h, return -1; } free(cat); - } else { + } else if (ms->flags & MAGIC_MIME_TYPE) { if (file_printf(ms, "application/CDFV2") == -1) return -1; } @@ -318,19 +314,19 @@ cdf_file_summary_info(struct magic_set *ms, const cdf_header_t *h, case 2: if (file_printf(ms, ", Os: Windows, Version %d.%d", si.si_os_version & 0xff, - (uint32_t)si.si_os_version >> 8) == -1) + CAST(uint32_t, si.si_os_version) >> 8) == -1) return -2; break; case 1: if (file_printf(ms, ", Os: MacOS, Version %d.%d", - (uint32_t)si.si_os_version >> 8, + CAST(uint32_t, si.si_os_version) >> 8, si.si_os_version & 0xff) == -1) return -2; break; default: if (file_printf(ms, ", Os %d, Version: %d.%d", si.si_os, si.si_os_version & 0xff, - (uint32_t)si.si_os_version >> 8) == -1) + CAST(uint32_t, si.si_os_version) >> 8) == -1) return -2; break; } @@ -353,11 +349,11 @@ cdf_file_summary_info(struct magic_set *ms, const cdf_header_t *h, #ifdef notdef private char * format_clsid(char *buf, size_t len, const uint64_t uuid[2]) { - snprintf(buf, len, "%.8" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.4" + snprintf(buf, len, "%.8" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.12" PRIx64, (uuid[0] >> 32) & (uint64_t)0x000000000ffffffffULL, (uuid[0] >> 16) & (uint64_t)0x0000000000000ffffULL, - (uuid[0] >> 0) & (uint64_t)0x0000000000000ffffULL, + (uuid[0] >> 0) & (uint64_t)0x0000000000000ffffULL, (uuid[1] >> 48) & (uint64_t)0x0000000000000ffffULL, (uuid[1] >> 0) & (uint64_t)0x0000fffffffffffffULL); return buf; @@ -406,7 +402,7 @@ cdf_check_summary_info(struct magic_set *ms, const cdf_info_t *info, for (j = 0; str == NULL && j < dir->dir_len; j++) { d = &dir->dir_tab[j]; for (k = 0; k < sizeof(name); k++) - name[k] = (char)cdf_tole2(d->d_name[k]); + name[k] = CAST(char, cdf_tole2(d->d_name[k])); str = cdf_app_to_mime(name, NOTMIME(ms) ? name2desc : name2mime); } @@ -416,7 +412,7 @@ cdf_check_summary_info(struct magic_set *ms, const cdf_info_t *info, return -1; i = 1; } - } else { + } else if (ms->flags & MAGIC_MIME_TYPE) { if (str == NULL) str = "vnd.ms-office"; if (file_printf(ms, "application/%s", str) == -1) @@ -436,7 +432,7 @@ private struct sinfo { const char *sections[5]; const int types[5]; } sectioninfo[] = { - { "Encrypted", "encrypted", + { "Encrypted", "encrypted", { "EncryptedPackage", "EncryptedSummary", NULL, NULL, NULL, @@ -448,7 +444,7 @@ private struct sinfo { }, }, - { "QuickBooks", "quickbooks", + { "QuickBooks", "quickbooks", { #if 0 "TaxForms", "PDFTaxForms", "modulesInBackup", @@ -527,7 +523,7 @@ cdf_file_dir_info(struct magic_set *ms, const cdf_dir_t *dir) if (NOTMIME(ms)) { if (file_printf(ms, "CDFV2 %s", si->name) == -1) return -1; - } else { + } else if (ms->flags & MAGIC_MIME_TYPE) { if (file_printf(ms, "application/%s", si->mime) == -1) return -1; } @@ -540,7 +536,7 @@ protected int file_trycdf(struct magic_set *ms, const struct buffer *b) { int fd = b->fd; - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; cdf_info_t info; cdf_header_t h; @@ -614,7 +610,7 @@ file_trycdf(struct magic_set *ms, const struct buffer *b) if (file_printf(ms, "Hangul (Korean) Word Processor File 5.x") == -1) return -1; - } else { + } else if (ms->flags & MAGIC_MIME_TYPE) { if (file_printf(ms, "application/x-hwp") == -1) return -1; } @@ -661,19 +657,20 @@ out2: out1: free(sat.sat_tab); out0: - if (i == -1) { - if (NOTMIME(ms)) { + /* If we handled it already, return */ + if (i != -1) + return i; + /* Provide a default handler */ + if (NOTMIME(ms)) { if (file_printf(ms, "Composite Document File V2 Document") == -1) - return -1; - if (*expn) - if (file_printf(ms, ", %s", expn) == -1) return -1; - } else { + if (*expn) + if (file_printf(ms, ", %s", expn) == -1) + return -1; + } else if (ms->flags & MAGIC_MIME_TYPE) { if (file_printf(ms, "application/CDFV2") == -1) - return -1; - } - i = 1; + return -1; } - return i; + return 1; } diff --git a/src/readelf.c b/src/readelf.c index db0b35a506a7..ef61d4cd600f 100644 --- a/src/readelf.c +++ b/src/readelf.c @@ -1,7 +1,7 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -11,7 +11,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -27,7 +27,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: readelf.c,v 1.144 2018/07/08 23:37:33 christos Exp $") +FILE_RCSID("@(#)$File: readelf.c,v 1.165 2019/05/07 02:27:11 christos Exp $") #endif #ifdef BUILTIN_ELF @@ -69,7 +69,7 @@ toomany(struct magic_set *ms, const char *name, uint16_t num) { if (file_printf(ms, ", too many %s (%u)", name, num) == -1) return -1; - return 0; + return 1; } private uint16_t @@ -85,7 +85,7 @@ getu16(int swap, uint16_t value) retval.c[0] = tmpval.c[1]; retval.c[1] = tmpval.c[0]; - + return retval.ui; } else return value; @@ -106,7 +106,7 @@ getu32(int swap, uint32_t value) retval.c[1] = tmpval.c[2]; retval.c[2] = tmpval.c[1]; retval.c[3] = tmpval.c[0]; - + return retval.ui; } else return value; @@ -131,7 +131,7 @@ getu64(int swap, uint64_t value) retval.c[5] = tmpval.c[2]; retval.c[6] = tmpval.c[1]; retval.c[7] = tmpval.c[0]; - + return retval.ui; } else return value; @@ -262,7 +262,10 @@ static const size_t prpsoffsets32[] = { 84, /* SunOS 5.x (short name) */ 44, /* Linux (command line) */ - 28, /* Linux 2.0.36 (short name) */ + 28, /* Linux (short name) */ + + 48, /* Linux PowerPC (command line) */ + 32, /* Linux PowerPC (short name) */ 8, /* FreeBSD */ }; @@ -282,8 +285,8 @@ static const size_t prpsoffsets64[] = { 16, /* FreeBSD, 64-bit */ }; -#define NOFFSETS32 (sizeof(prpsoffsets32) / sizeof(prpsoffsets32[0])) -#define NOFFSETS64 (sizeof(prpsoffsets64) / sizeof(prpsoffsets64[0])) +#define NOFFSETS32 __arraycount(prpsoffsets32) +#define NOFFSETS64 __arraycount(prpsoffsets64) #define NOFFSETS (clazz == ELFCLASS32 ? NOFFSETS32 : NOFFSETS64) @@ -352,6 +355,11 @@ dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off, off_t ph_off = off; int ph_num = num; + if (num == 0) { + if (file_printf(ms, ", no program header") == -1) + return -1; + return 0; + } if (size != xph_sizeof) { if (file_printf(ms, ", corrupted program header size") == -1) return -1; @@ -388,9 +396,9 @@ dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off, } offset = 0; for (;;) { - if (offset >= (size_t)bufsize) + if (offset >= CAST(size_t, bufsize)) break; - offset = donote(ms, nbuf, offset, (size_t)bufsize, + offset = donote(ms, nbuf, offset, CAST(size_t, bufsize), clazz, swap, 4, flags, notecount, fd, ph_off, ph_num, fsize); if (offset == 0) @@ -402,7 +410,7 @@ dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off, } #endif -static void +static int do_note_netbsd_version(struct magic_set *ms, int swap, void *v) { uint32_t desc; @@ -410,7 +418,7 @@ do_note_netbsd_version(struct magic_set *ms, int swap, void *v) desc = elf_getu32(swap, desc); if (file_printf(ms, ", for NetBSD") == -1) - return; + return -1; /* * The version number used to be stuck as 199905, and was thus * basically content-free. Newer versions of NetBSD have fixed @@ -430,24 +438,25 @@ do_note_netbsd_version(struct magic_set *ms, int swap, void *v) uint32_t ver_maj = desc / 100000000; if (file_printf(ms, " %u.%u", ver_maj, ver_min) == -1) - return; + return -1; if (ver_rel == 0 && ver_patch != 0) { if (file_printf(ms, ".%u", ver_patch) == -1) - return; + return -1; } else if (ver_rel != 0) { while (ver_rel > 26) { if (file_printf(ms, "Z") == -1) - return; + return -1; ver_rel -= 26; } if (file_printf(ms, "%c", 'A' + ver_rel - 1) == -1) - return; + return -1; } } + return 0; } -static void +static int do_note_freebsd_version(struct magic_set *ms, int swap, void *v) { uint32_t desc; @@ -455,71 +464,72 @@ do_note_freebsd_version(struct magic_set *ms, int swap, void *v) memcpy(&desc, v, sizeof(desc)); desc = elf_getu32(swap, desc); if (file_printf(ms, ", for FreeBSD") == -1) - return; + return -1; /* * Contents is __FreeBSD_version, whose relation to OS * versions is defined by a huge table in the Porter's * Handbook. This is the general scheme: - * + * * Releases: * Mmp000 (before 4.10) * Mmi0p0 (before 5.0) * Mmm0p0 - * + * * Development branches: * Mmpxxx (before 4.6) * Mmp1xx (before 4.10) * Mmi1xx (before 5.0) * M000xx (pre-M.0) * Mmm1xx - * + * * M = major version * m = minor version * i = minor version increment (491000 -> 4.10) * p = patchlevel * x = revision - * + * * The first release of FreeBSD to use ELF by default * was version 3.0. */ if (desc == 460002) { if (file_printf(ms, " 4.6.2") == -1) - return; + return -1; } else if (desc < 460100) { if (file_printf(ms, " %d.%d", desc / 100000, desc / 10000 % 10) == -1) - return; + return -1; if (desc / 1000 % 10 > 0) if (file_printf(ms, ".%d", desc / 1000 % 10) == -1) - return; + return -1; if ((desc % 1000 > 0) || (desc % 100000 == 0)) if (file_printf(ms, " (%d)", desc) == -1) - return; + return -1; } else if (desc < 500000) { if (file_printf(ms, " %d.%d", desc / 100000, desc / 10000 % 10 + desc / 1000 % 10) == -1) - return; + return -1; if (desc / 100 % 10 > 0) { if (file_printf(ms, " (%d)", desc) == -1) - return; + return -1; } else if (desc / 10 % 10 > 0) { if (file_printf(ms, ".%d", desc / 10 % 10) == -1) - return; + return -1; } } else { if (file_printf(ms, " %d.%d", desc / 100000, desc / 1000 % 100) == -1) - return; + return -1; if ((desc / 100 % 10 > 0) || (desc % 100000 / 100 == 0)) { if (file_printf(ms, " (%d)", desc) == -1) - return; + return -1; } else if (desc / 10 % 10 > 0) { if (file_printf(ms, ".%d", desc / 10 % 10) == -1) - return; + return -1; } } + return 0; } private int @@ -528,7 +538,7 @@ do_bid_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, int swap __attribute__((__unused__)), uint32_t namesz, uint32_t descsz, size_t noff, size_t doff, int *flags) { - if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 && + if (namesz == 4 && strcmp(RCAST(char *, &nbuf[noff]), "GNU") == 0 && type == NT_GNU_BUILD_ID && (descsz >= 4 && descsz <= 20)) { uint8_t desc[20]; const char *btype; @@ -549,18 +559,18 @@ do_bid_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, break; } if (file_printf(ms, ", BuildID[%s]=", btype) == -1) - return 1; + return -1; memcpy(desc, &nbuf[doff], descsz); for (i = 0; i < descsz; i++) if (file_printf(ms, "%02x", desc[i]) == -1) - return 1; + return -1; return 1; } - if (namesz == 4 && strcmp((char *)&nbuf[noff], "Go") == 0 && + if (namesz == 4 && strcmp(RCAST(char *, &nbuf[noff]), "Go") == 0 && type == NT_GO_BUILD_ID && descsz < 128) { - if (file_printf(ms, ", Go BuildID=%s", - (char *)&nbuf[doff]) == -1) - return 1; + if (file_printf(ms, ", Go BuildID=%.*s", + CAST(int, descsz), RCAST(char *, &nbuf[doff])) == -1) + return -1; return 1; } return 0; @@ -571,88 +581,95 @@ do_os_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, int swap, uint32_t namesz, uint32_t descsz, size_t noff, size_t doff, int *flags) { - if (namesz == 5 && strcmp((char *)&nbuf[noff], "SuSE") == 0 && - type == NT_GNU_VERSION && descsz == 2) { - *flags |= FLAGS_DID_OS_NOTE; - file_printf(ms, ", for SuSE %d.%d", nbuf[doff], nbuf[doff + 1]); + const char *name = RCAST(const char *, &nbuf[noff]); + + if (namesz == 5 && strcmp(name, "SuSE") == 0 && + type == NT_GNU_VERSION && descsz == 2) { + *flags |= FLAGS_DID_OS_NOTE; + if (file_printf(ms, ", for SuSE %d.%d", nbuf[doff], + nbuf[doff + 1]) == -1) + return -1; return 1; } - if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 && + if (namesz == 4 && strcmp(name, "GNU") == 0 && type == NT_GNU_VERSION && descsz == 16) { uint32_t desc[4]; memcpy(desc, &nbuf[doff], sizeof(desc)); *flags |= FLAGS_DID_OS_NOTE; if (file_printf(ms, ", for GNU/") == -1) - return 1; + return -1; switch (elf_getu32(swap, desc[0])) { case GNU_OS_LINUX: if (file_printf(ms, "Linux") == -1) - return 1; + return -1; break; case GNU_OS_HURD: if (file_printf(ms, "Hurd") == -1) - return 1; + return -1; break; case GNU_OS_SOLARIS: if (file_printf(ms, "Solaris") == -1) - return 1; + return -1; break; case GNU_OS_KFREEBSD: if (file_printf(ms, "kFreeBSD") == -1) - return 1; + return -1; break; case GNU_OS_KNETBSD: if (file_printf(ms, "kNetBSD") == -1) - return 1; + return -1; break; default: if (file_printf(ms, "<unknown>") == -1) - return 1; + return -1; } if (file_printf(ms, " %d.%d.%d", elf_getu32(swap, desc[1]), elf_getu32(swap, desc[2]), elf_getu32(swap, desc[3])) == -1) - return 1; + return -1; return 1; } - if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) { + if (namesz == 7 && strcmp(name, "NetBSD") == 0) { if (type == NT_NETBSD_VERSION && descsz == 4) { *flags |= FLAGS_DID_OS_NOTE; - do_note_netbsd_version(ms, swap, &nbuf[doff]); + if (do_note_netbsd_version(ms, swap, &nbuf[doff]) == -1) + return -1; return 1; } } - if (namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0) { + if (namesz == 8 && strcmp(name, "FreeBSD") == 0) { if (type == NT_FREEBSD_VERSION && descsz == 4) { *flags |= FLAGS_DID_OS_NOTE; - do_note_freebsd_version(ms, swap, &nbuf[doff]); + if (do_note_freebsd_version(ms, swap, &nbuf[doff]) + == -1) + return -1; return 1; } } - if (namesz == 8 && strcmp((char *)&nbuf[noff], "OpenBSD") == 0 && + if (namesz == 8 && strcmp(name, "OpenBSD") == 0 && type == NT_OPENBSD_VERSION && descsz == 4) { *flags |= FLAGS_DID_OS_NOTE; if (file_printf(ms, ", for OpenBSD") == -1) - return 1; + return -1; /* Content of note is always 0 */ return 1; } - if (namesz == 10 && strcmp((char *)&nbuf[noff], "DragonFly") == 0 && + if (namesz == 10 && strcmp(name, "DragonFly") == 0 && type == NT_DRAGONFLY_VERSION && descsz == 4) { uint32_t desc; *flags |= FLAGS_DID_OS_NOTE; if (file_printf(ms, ", for DragonFly") == -1) - return 1; + return -1; memcpy(&desc, &nbuf[doff], sizeof(desc)); desc = elf_getu32(swap, desc); if (file_printf(ms, " %d.%d.%d", desc / 100000, desc / 10000 % 10, desc % 10000) == -1) - return 1; + return -1; return 1; } return 0; @@ -663,7 +680,9 @@ do_pax_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, int swap, uint32_t namesz, uint32_t descsz, size_t noff, size_t doff, int *flags) { - if (namesz == 4 && strcmp((char *)&nbuf[noff], "PaX") == 0 && + const char *name = RCAST(const char *, &nbuf[noff]); + + if (namesz == 4 && strcmp(name, "PaX") == 0 && type == NT_NETBSD_PAX && descsz == 4) { static const char *pax[] = { "+mprotect", @@ -682,14 +701,14 @@ do_pax_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, desc = elf_getu32(swap, desc); if (desc && file_printf(ms, ", PaX: ") == -1) - return 1; + return -1; for (i = 0; i < __arraycount(pax); i++) { - if (((1 << (int)i) & desc) == 0) + if (((1 << CAST(int, i)) & desc) == 0) continue; if (file_printf(ms, "%s%s", did++ ? "," : "", pax[i]) == -1) - return 1; + return -1; } return 1; } @@ -702,6 +721,8 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, size_t noff, size_t doff, int *flags, size_t size, int clazz) { #ifdef ELFCORE + const char *name = RCAST(const char *, &nbuf[noff]); + int os_style = -1; /* * Sigh. The 2.0.36 kernel in Debian 2.1, at @@ -717,16 +738,16 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, * doesn't include the terminating null in the * name.... */ - if ((namesz == 4 && strncmp((char *)&nbuf[noff], "CORE", 4) == 0) || - (namesz == 5 && strcmp((char *)&nbuf[noff], "CORE") == 0)) { + if ((namesz == 4 && strncmp(name, "CORE", 4) == 0) || + (namesz == 5 && strcmp(name, "CORE") == 0)) { os_style = OS_STYLE_SVR4; - } + } - if ((namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0)) { + if ((namesz == 8 && strcmp(name, "FreeBSD") == 0)) { os_style = OS_STYLE_FREEBSD; } - if ((namesz >= 11 && strncmp((char *)&nbuf[noff], "NetBSD-CORE", 11) + if ((namesz >= 11 && strncmp(name, "NetBSD-CORE", 11) == 0)) { os_style = OS_STYLE_NETBSD; } @@ -734,7 +755,7 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, if (os_style != -1 && (*flags & FLAGS_DID_CORE_STYLE) == 0) { if (file_printf(ms, ", %s-style", os_style_names[os_style]) == -1) - return 1; + return -1; *flags |= FLAGS_DID_CORE_STYLE; *flags |= os_style; } @@ -745,26 +766,48 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, char sbuf[512]; struct NetBSD_elfcore_procinfo pi; memset(&pi, 0, sizeof(pi)); - memcpy(&pi, nbuf + doff, descsz); + memcpy(&pi, nbuf + doff, MIN(descsz, sizeof(pi))); if (file_printf(ms, ", from '%.31s', pid=%u, uid=%u, " "gid=%u, nlwps=%u, lwp=%u (signal %u/code %u)", file_printable(sbuf, sizeof(sbuf), - CAST(char *, pi.cpi_name)), - elf_getu32(swap, (uint32_t)pi.cpi_pid), + RCAST(char *, pi.cpi_name), sizeof(pi.cpi_name)), + elf_getu32(swap, CAST(uint32_t, pi.cpi_pid)), elf_getu32(swap, pi.cpi_euid), elf_getu32(swap, pi.cpi_egid), elf_getu32(swap, pi.cpi_nlwps), - elf_getu32(swap, (uint32_t)pi.cpi_siglwp), + elf_getu32(swap, CAST(uint32_t, pi.cpi_siglwp)), elf_getu32(swap, pi.cpi_signo), elf_getu32(swap, pi.cpi_sigcode)) == -1) - return 1; + return -1; *flags |= FLAGS_DID_CORE; return 1; } break; + case OS_STYLE_FREEBSD: + if (type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) { + size_t argoff, pidoff; + + if (clazz == ELFCLASS32) + argoff = 4 + 4 + 17; + else + argoff = 4 + 4 + 8 + 17; + if (file_printf(ms, ", from '%.80s'", nbuf + doff + + argoff) == -1) + return -1; + pidoff = argoff + 81 + 2; + if (doff + pidoff + 4 <= size) { + if (file_printf(ms, ", pid=%u", + elf_getu32(swap, *RCAST(uint32_t *, (nbuf + + doff + pidoff)))) == -1) + return -1; + } + *flags |= FLAGS_DID_CORE; + } + break; + default: if (type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) { size_t i, j; @@ -847,8 +890,8 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, i = k; } - cname = (unsigned char *) - &nbuf[doff + prpsoffsets(i)]; + cname = CAST(unsigned char *, + &nbuf[doff + prpsoffsets(i)]); for (cp = cname; cp < nbuf + size && *cp && isprint(*cp); cp++) continue; @@ -859,8 +902,8 @@ do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, while (cp > cname && isspace(cp[-1])) cp--; if (file_printf(ms, ", from '%.*s'", - (int)(cp - cname), cname) == -1) - return 1; + CAST(int, cp - cname), cname) == -1) + return -1; *flags |= FLAGS_DID_CORE; return 1; @@ -886,7 +929,8 @@ get_offset_from_virtaddr(struct magic_set *ms, int swap, int clazz, int fd, * virtual address in which the "virtaddr" belongs to. */ for ( ; num; num--) { - if (pread(fd, xph_addr, xph_sizeof, off) < (ssize_t)xph_sizeof) { + if (pread(fd, xph_addr, xph_sizeof, off) < + CAST(ssize_t, xph_sizeof)) { file_badread(ms); return -1; } @@ -916,7 +960,8 @@ get_string_on_virtaddr(struct magic_set *ms, offset = get_offset_from_virtaddr(ms, swap, clazz, fd, ph_off, ph_num, fsize, virtaddr); - if ((buflen = pread(fd, buf, CAST(size_t, buflen), offset)) <= 0) { + if (offset < 0 || + (buflen = pread(fd, buf, CAST(size_t, buflen), offset)) <= 0) { file_badread(ms); return 0; } @@ -925,7 +970,7 @@ get_string_on_virtaddr(struct magic_set *ms, /* We expect only printable characters, so return if buffer contains * non-printable character before the '\0' or just '\0'. */ - for (bptr = buf; *bptr && isprint((unsigned char)*bptr); bptr++) + for (bptr = buf; *bptr && isprint(CAST(unsigned char, *bptr)); bptr++) continue; if (*bptr != '\0') return 0; @@ -1027,13 +1072,13 @@ do_auxv_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, if (buflen == 0) continue; - + if (file_printf(ms, ", %s: '%s'", tag, buf) == -1) - return 0; + return -1; } else { - if (file_printf(ms, ", %s: %d", tag, (int) xauxv_val) - == -1) - return 0; + if (file_printf(ms, ", %s: %d", tag, + CAST(int, xauxv_val)) == -1) + return -1; } } return 1; @@ -1062,7 +1107,7 @@ dodynamic(struct magic_set *ms, void *vbuf, size_t offset, size_t size, switch (xdh_tag) { case DT_FLAGS_1: - if (xdh_val == DF_1_PIE) + if (xdh_val & DF_1_PIE) ms->mode |= 0111; else ms->mode &= ~0111; @@ -1110,14 +1155,16 @@ donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, } if (namesz & 0x80000000) { - file_printf(ms, ", bad note name size %#lx", - CAST(unsigned long, namesz)); + if (file_printf(ms, ", bad note name size %#lx", + CAST(unsigned long, namesz)) == -1) + return -1; return 0; } if (descsz & 0x80000000) { - file_printf(ms, ", bad note description size %#lx", - CAST(unsigned long, descsz)); + if (file_printf(ms, ", bad note description size %#lx", + CAST(unsigned long, descsz)) == -1) + return -1; return 0; } @@ -1151,7 +1198,7 @@ donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, namesz, descsz, noff, doff, flags)) return offset; } - + if ((*flags & FLAGS_DID_NETBSD_PAX) == 0) { if (do_pax_note(ms, nbuf, xnh_type, swap, namesz, descsz, noff, doff, flags)) @@ -1171,7 +1218,7 @@ donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, return offset; } - if (namesz == 7 && strcmp(CAST(char *, &nbuf[noff]), "NetBSD") == 0) { + if (namesz == 7 && strcmp(RCAST(char *, &nbuf[noff]), "NetBSD") == 0) { int descw, flag; const char *str, *tag; if (descsz > 100) @@ -1202,7 +1249,7 @@ donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, if (*flags & flag) return offset; - str = CAST(const char *, &nbuf[doff]); + str = RCAST(const char *, &nbuf[doff]); descw = CAST(int, descsz); *flags |= flag; file_printf(ms, ", %s: %.*s", tag, descw, str); @@ -1278,6 +1325,11 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num, char name[50]; ssize_t namesize; + if (num == 0) { + if (file_printf(ms, ", no section header") == -1) + return -1; + return 0; + } if (size != xsh_sizeof) { if (file_printf(ms, ", corrupted section header size") == -1) return -1; @@ -1343,7 +1395,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num, CAST(uintmax_t, xsh_size), CAST(uintmax_t, fsize)) == -1) return -1; - return 0; + return 0; } if ((nbuf = malloc(xsh_size)) == NULL) { file_error(ms, errno, "Cannot allocate memory" @@ -1428,7 +1480,7 @@ doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num, return -1; break; } - // gnu attributes + // gnu attributes #endif break; } @@ -1548,7 +1600,12 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, char interp[BUFSIZ]; ssize_t bufsize; size_t offset, align, len; - + + if (num == 0) { + if (file_printf(ms, ", no program header") == -1) + return -1; + return 0; + } if (size != xph_sizeof) { if (file_printf(ms, ", corrupted program header size") == -1) return -1; @@ -1558,7 +1615,7 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, interp[0] = '\0'; for ( ; num; num--) { int doread; - if (pread(fd, xph_addr, xph_sizeof, off) < + if (pread(fd, xph_addr, xph_sizeof, off) < CAST(ssize_t, xph_sizeof)) { file_badread(ms); return -1; @@ -1579,7 +1636,7 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, continue; if (((align = xph_align) & 0x80000000UL) != 0 || align < 4) { - if (file_printf(ms, + if (file_printf(ms, ", invalid note alignment %#lx", CAST(unsigned long, align)) == -1) return -1; @@ -1613,8 +1670,10 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, switch (xph_type) { case PT_DYNAMIC: offset = 0; + // Let DF_1 determine if we are PIE or not. + ms->mode &= ~0111; for (;;) { - if (offset >= (size_t)bufsize) + if (offset >= CAST(size_t, bufsize)) break; offset = dodynamic(ms, nbuf, offset, CAST(size_t, bufsize), clazz, swap); @@ -1626,7 +1685,7 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, case PT_INTERP: if (bufsize && nbuf[0]) { nbuf[bufsize - 1] = '\0'; - memcpy(interp, nbuf, bufsize); + memcpy(interp, nbuf, CAST(size_t, bufsize)); } else strlcpy(interp, "*empty*", sizeof(interp)); break; @@ -1637,7 +1696,7 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, */ offset = 0; for (;;) { - if (offset >= (size_t)bufsize) + if (offset >= CAST(size_t, bufsize)) break; offset = donote(ms, nbuf, offset, CAST(size_t, bufsize), clazz, swap, align, @@ -1655,7 +1714,8 @@ dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, return -1; if (interp[0]) if (file_printf(ms, ", interpreter %s", - file_printable(ibuf, sizeof(ibuf), interp)) == -1) + file_printable(ibuf, sizeof(ibuf), interp, sizeof(interp))) + == -1) return -1; return 0; } @@ -1665,7 +1725,7 @@ protected int file_tryelf(struct magic_set *ms, const struct buffer *b) { int fd = b->fd; - const unsigned char *buf = b->fbuf; + const unsigned char *buf = CAST(const unsigned char *, b->fbuf); size_t nbytes = b->flen; union { int32_t l; @@ -1674,6 +1734,7 @@ file_tryelf(struct magic_set *ms, const struct buffer *b) int clazz; int swap; struct stat st; + const struct stat *stp; off_t fsize; int flags = 0; Elf32_Ehdr elf32hdr; @@ -1687,7 +1748,8 @@ file_tryelf(struct magic_set *ms, const struct buffer *b) * file locations and thus file(1) cannot determine it from easily. * Instead we traverse thru all section headers until a symbol table * one is found or else the binary is stripped. - * Return immediately if it's not ELF (so we avoid pipe2file unless needed). + * Return immediately if it's not ELF (so we avoid pipe2file unless + * needed). */ if (buf[EI_MAG0] != ELFMAG0 || (buf[EI_MAG1] != ELFMAG1 && buf[EI_MAG1] != OLFMAG1) @@ -1701,12 +1763,25 @@ file_tryelf(struct magic_set *ms, const struct buffer *b) && (errno == ESPIPE)) fd = file_pipe2file(ms, fd, buf, nbytes); - if (fstat(fd, &st) == -1) { - file_badread(ms); + if (fd == -1) { + file_badread(ms); return -1; } - if (S_ISREG(st.st_mode) || st.st_size != 0) - fsize = st.st_size; + + stp = &b->st; + /* + * b->st.st_size != 0 if previous fstat() succeeded, + * which is likely, we can avoid extra stat() call. + */ + if (b->st.st_size == 0) { + stp = &st; + if (fstat(fd, &st) == -1) { + file_badread(ms); + return -1; + } + } + if (S_ISREG(stp->st_mode) || stp->st_size != 0) + fsize = stp->st_size; else fsize = SIZE_UNKNOWN; diff --git a/src/readelf.h b/src/readelf.h index 6ae63f2d950d..809d3f7573bd 100644 --- a/src/readelf.h +++ b/src/readelf.h @@ -1,7 +1,7 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -11,7 +11,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -304,7 +304,7 @@ typedef struct { #define GNU_OS_KNETBSD 4 /* - * GNU Hardware capability information + * GNU Hardware capability information * word[0]: Number of entries * word[1]: Bitmask of enabled entries * Followed by a byte id, and a NUL terminated string per entry @@ -313,7 +313,7 @@ typedef struct { /* * GNU Build ID generated by ld - * 160 bit SHA1 [default] + * 160 bit SHA1 [default] * 128 bit md5 or uuid */ #define NT_GNU_BUILD_ID 3 diff --git a/src/seccomp.c b/src/seccomp.c index a5abb4a159f9..1b9d9b855937 100644 --- a/src/seccomp.c +++ b/src/seccomp.c @@ -27,7 +27,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: seccomp.c,v 1.6 2018/06/26 20:29:29 christos Exp $") +FILE_RCSID("@(#)$File: seccomp.c,v 1.8 2019/02/24 18:12:04 christos Exp $") #endif /* lint */ #if HAVE_LIBSECCOMP @@ -126,14 +126,14 @@ enable_sandbox_basic(void) DENY_RULE (socket); // ... - + // applying filter... if (seccomp_load (ctx) == -1) goto out; // free ctx after the filter has been loaded into the kernel seccomp_release(ctx); return 0; - + out: seccomp_release(ctx); return -1; @@ -151,7 +151,7 @@ enable_sandbox_full(void) if (prctl(PR_SET_DUMPABLE, 0, 0, 0, 0) == -1) return -1; - + // initialize the filter ctx = seccomp_init(SCMP_ACT_KILL); if (ctx == NULL) @@ -163,10 +163,10 @@ enable_sandbox_full(void) ALLOW_RULE(dup2); ALLOW_RULE(exit); ALLOW_RULE(exit_group); - ALLOW_RULE(fcntl); - ALLOW_RULE(fcntl64); + ALLOW_RULE(fcntl); + ALLOW_RULE(fcntl64); ALLOW_RULE(fstat); - ALLOW_RULE(fstat64); + ALLOW_RULE(fstat64); ALLOW_RULE(getdents); #ifdef __NR_getdents64 ALLOW_RULE(getdents64); @@ -176,6 +176,7 @@ enable_sandbox_full(void) ALLOW_RULE(_llseek); ALLOW_RULE(lstat); ALLOW_RULE(lstat64); + ALLOW_RULE(madvise); ALLOW_RULE(mmap); ALLOW_RULE(mmap2); ALLOW_RULE(mprotect); diff --git a/src/softmagic.c b/src/softmagic.c index 9a5db9d3e2b3..2edae41df4da 100644 --- a/src/softmagic.c +++ b/src/softmagic.c @@ -32,7 +32,7 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: softmagic.c,v 1.262 2018/06/22 20:39:50 christos Exp $") +FILE_RCSID("@(#)$File: softmagic.c,v 1.286 2019/05/17 02:24:59 christos Exp $") #endif /* lint */ #include "magic.h" @@ -45,11 +45,11 @@ FILE_RCSID("@(#)$File: softmagic.c,v 1.262 2018/06/22 20:39:50 christos Exp $") private int match(struct magic_set *, struct magic *, uint32_t, const struct buffer *, size_t, int, int, int, uint16_t *, - uint16_t *, int *, int *, int *); + uint16_t *, int *, int *, int *, int *); private int mget(struct magic_set *, struct magic *, const struct buffer *, - const unsigned char *, size_t, + const unsigned char *, size_t, size_t, unsigned int, int, int, int, uint16_t *, - uint16_t *, int *, int *, int *); + uint16_t *, int *, int *, int *, int *); private int msetoffset(struct magic_set *, struct magic *, struct buffer *, const struct buffer *, size_t, unsigned int); private int magiccheck(struct magic_set *, struct magic *); @@ -67,24 +67,46 @@ private int cvt_16(union VALUETYPE *, const struct magic *); private int cvt_32(union VALUETYPE *, const struct magic *); private int cvt_64(union VALUETYPE *, const struct magic *); -#define OFFSET_OOB(n, o, i) ((n) < (uint32_t)(o) || (i) > ((n) - (o))) -#define BE64(p) (((uint64_t)(p)->hq[0]<<56)|((uint64_t)(p)->hq[1]<<48)| \ - ((uint64_t)(p)->hq[2]<<40)|((uint64_t)(p)->hq[3]<<32)| \ - ((uint64_t)(p)->hq[4]<<24)|((uint64_t)(p)->hq[5]<<16)| \ - ((uint64_t)(p)->hq[6]<<8)|((uint64_t)(p)->hq[7])) -#define LE64(p) (((uint64_t)(p)->hq[7]<<56)|((uint64_t)(p)->hq[6]<<48)| \ - ((uint64_t)(p)->hq[5]<<40)|((uint64_t)(p)->hq[4]<<32)| \ - ((uint64_t)(p)->hq[3]<<24)|((uint64_t)(p)->hq[2]<<16)| \ - ((uint64_t)(p)->hq[1]<<8)|((uint64_t)(p)->hq[0])) -#define LE32(p) (((uint32_t)(p)->hl[3]<<24)|((uint32_t)(p)->hl[2]<<16)| \ - ((uint32_t)(p)->hl[1]<<8)|((uint32_t)(p)->hl[0])) -#define BE32(p) (((uint32_t)(p)->hl[0]<<24)|((uint32_t)(p)->hl[1]<<16)| \ - ((uint32_t)(p)->hl[2]<<8)|((uint32_t)(p)->hl[3])) -#define ME32(p) (((uint32_t)(p)->hl[1]<<24)|((uint32_t)(p)->hl[0]<<16)| \ - ((uint32_t)(p)->hl[3]<<8)|((uint32_t)(p)->hl[2])) -#define BE16(p) (((uint16_t)(p)->hs[0]<<8)|((uint16_t)(p)->hs[1])) -#define LE16(p) (((uint16_t)(p)->hs[1]<<8)|((uint16_t)(p)->hs[0])) -#define SEXT(s,v,p) ((s)?(intmax_t)(int##v##_t)(p):(intmax_t)(uint##v##_t)(p)) +#define OFFSET_OOB(n, o, i) ((n) < CAST(uint32_t, (o)) || (i) > ((n) - (o))) +#define BE64(p) ( \ + (CAST(uint64_t, (p)->hq[0])<<56)| \ + (CAST(uint64_t, (p)->hq[1])<<48)| \ + (CAST(uint64_t, (p)->hq[2])<<40)| \ + (CAST(uint64_t, (p)->hq[3])<<32)| \ + (CAST(uint64_t, (p)->hq[4])<<24)| \ + (CAST(uint64_t, (p)->hq[5])<<16)| \ + (CAST(uint64_t, (p)->hq[6])<<8)| \ + (CAST(uint64_t, (p)->hq[7]))) +#define LE64(p) ( \ + (CAST(uint64_t, (p)->hq[7])<<56)| \ + (CAST(uint64_t, (p)->hq[6])<<48)| \ + (CAST(uint64_t, (p)->hq[5])<<40)| \ + (CAST(uint64_t, (p)->hq[4])<<32)| \ + (CAST(uint64_t, (p)->hq[3])<<24)| \ + (CAST(uint64_t, (p)->hq[2])<<16)| \ + (CAST(uint64_t, (p)->hq[1])<<8)| \ + (CAST(uint64_t, (p)->hq[0]))) +#define LE32(p) ( \ + (CAST(uint32_t, (p)->hl[3])<<24)| \ + (CAST(uint32_t, (p)->hl[2])<<16)| \ + (CAST(uint32_t, (p)->hl[1])<<8)| \ + (CAST(uint32_t, (p)->hl[0]))) +#define BE32(p) ( \ + (CAST(uint32_t, (p)->hl[0])<<24)| \ + (CAST(uint32_t, (p)->hl[1])<<16)| \ + (CAST(uint32_t, (p)->hl[2])<<8)| \ + (CAST(uint32_t, (p)->hl[3]))) +#define ME32(p) ( \ + (CAST(uint32_t, (p)->hl[1])<<24)| \ + (CAST(uint32_t, (p)->hl[0])<<16)| \ + (CAST(uint32_t, (p)->hl[3])<<8)| \ + (CAST(uint32_t, (p)->hl[2]))) + +#define BE16(p) ((CAST(uint16_t, (p)->hs[0])<<8)|(CAST(uint16_t, (p)->hs[1]))) +#define LE16(p) ((CAST(uint16_t, (p)->hs[1])<<8)|(CAST(uint16_t, (p)->hs[0]))) +#define SEXT(s,v,p) ((s) ? \ + CAST(intmax_t, CAST(int##v##_t, p)) : \ + CAST(intmax_t, CAST(uint##v##_t, p))) /* * softmagic - lookup one file in parsed, in-memory copy of database @@ -111,7 +133,7 @@ file_softmagic(struct magic_set *ms, const struct buffer *b, for (ml = ms->mlist[0]->next; ml != ms->mlist[0]; ml = ml->next) if ((rv = match(ms, ml->magic, ml->nmagic, b, 0, mode, text, 0, indir_count, name_count, - &printed_something, &need_separator, NULL)) != 0) + &printed_something, &need_separator, NULL, NULL)) != 0) return rv; return 0; @@ -125,7 +147,12 @@ private const char * __attribute__((__format_arg__(3))) file_fmtcheck(struct magic_set *ms, const char *desc, const char *def, const char *file, size_t line) { - const char *ptr = fmtcheck(desc, def); + const char *ptr; + + if (strchr(desc, '%') == NULL) + return desc; + + ptr = fmtcheck(desc, def); if (ptr == def) file_magerror(ms, "%s, %" SIZE_T_FORMAT "u: format `%s' does not match" @@ -167,17 +194,25 @@ private int match(struct magic_set *ms, struct magic *magic, uint32_t nmagic, const struct buffer *b, size_t offset, int mode, int text, int flip, uint16_t *indir_count, uint16_t *name_count, - int *printed_something, int *need_separator, int *returnval) + int *printed_something, int *need_separator, int *returnval, + int *found_match) { uint32_t magindex = 0; unsigned int cont_level = 0; - int returnvalv = 0, e; /* if a match is found it is set to 1*/ + int found_matchv = 0; /* if a match is found it is set to 1*/ + int returnvalv = 0, e; int firstline = 1; /* a flag to print X\n X\n- X */ struct buffer bb; int print = (ms->flags & MAGIC_NODESC) == 0; + /* + * returnval can be 0 if a match is found, but there was no + * annotation to be printed. + */ if (returnval == NULL) returnval = &returnvalv; + if (found_match == NULL) + found_match = &found_matchv; if (file_check_mem(ms, cont_level) == -1) return -1; @@ -206,17 +241,21 @@ flush: ms->line = m->lineno; /* if main entry matches, print it... */ - switch (mget(ms, m, b, bb.fbuf, bb.flen, offset, cont_level, + switch (mget(ms, m, b, CAST(const unsigned char *, bb.fbuf), + bb.flen, offset, cont_level, mode, text, flip, indir_count, name_count, - printed_something, need_separator, returnval)) { + printed_something, need_separator, returnval, found_match)) + { case -1: return -1; case 0: flush = m->reln != '!'; break; default: - if (m->type == FILE_INDIRECT) + if (m->type == FILE_INDIRECT) { + *found_match = 1; *returnval = 1; + } switch (magiccheck(ms, m)) { case -1: @@ -238,7 +277,11 @@ flush: goto flush; } - if ((e = handle_annotation(ms, m, firstline)) != 0) { + if (*m->desc) + *found_match = 1; + + if ((e = handle_annotation(ms, m, firstline)) != 0) + { *need_separator = 1; *printed_something = 1; *returnval = 1; @@ -249,16 +292,16 @@ flush: * If we are going to print something, we'll need to print * a blank before we print something else. */ - if (*m->desc) { + if (print && *m->desc) { *need_separator = 1; *printed_something = 1; + *returnval = 1; if (print_sep(ms, firstline) == -1) return -1; + if (mprint(ms, m) == -1) + return -1; } - if (print && mprint(ms, m) == -1) - return -1; - switch (moffset(ms, m, &bb, &ms->c.li[cont_level].off)) { case -1: case 0: @@ -299,10 +342,11 @@ flush: continue; } #endif - switch (mget(ms, m, b, bb.fbuf, bb.flen, offset, + switch (mget(ms, m, b, CAST(const unsigned char *, + bb.fbuf), bb.flen, offset, cont_level, mode, text, flip, indir_count, name_count, printed_something, need_separator, - returnval)) { + returnval, found_match)) { case -1: return -1; case 0: @@ -311,8 +355,10 @@ flush: flush = 1; break; default: - if (m->type == FILE_INDIRECT) + if (m->type == FILE_INDIRECT) { + *found_match = 1; *returnval = 1; + } flush = 0; break; } @@ -337,6 +383,9 @@ flush: } else ms->c.li[cont_level].got_match = 1; + if (*m->desc) + *found_match = 1; + if ((e = handle_annotation(ms, m, firstline)) != 0) { *need_separator = 1; @@ -344,35 +393,36 @@ flush: *returnval = 1; return e; } - /* - * If we are going to print something, - * make sure that we have a separator first. - */ - if (*m->desc) { + if (print && *m->desc) { + /* + * This continuation matched. Print + * its message, with a blank before it + * if the previous item printed and + * this item isn't empty. + */ + /* + * If we are going to print something, + * make sure that we have a separator + * first. + */ if (!*printed_something) { *printed_something = 1; if (print_sep(ms, firstline) == -1) return -1; } - } - /* - * This continuation matched. Print - * its message, with a blank before it - * if the previous item printed and - * this item isn't empty. - */ - /* space if previous printed */ - if (*need_separator - && ((m->flag & NOSPACE) == 0) - && *m->desc) { - if (print && - file_printf(ms, " ") == -1) - return -1; + /* space if previous printed */ + if (*need_separator + && (m->flag & NOSPACE) == 0) { + if (file_printf(ms, " ") == -1) + return -1; + } + *returnval = 1; *need_separator = 0; + if (mprint(ms, m) == -1) + return -1; + *need_separator = 1; } - if (print && mprint(ms, m) == -1) - return -1; switch (moffset(ms, m, &bb, &ms->c.li[cont_level].off)) { @@ -385,9 +435,6 @@ flush: break; } - if (*m->desc) - *need_separator = 1; - /* * If we see any continuations * at a higher level, @@ -400,11 +447,13 @@ flush: } if (*printed_something) { firstline = 0; - if (print) - *returnval = 1; } - if ((ms->flags & MAGIC_CONTINUE) == 0 && *printed_something) { + if (*found_match) { + if ((ms->flags & MAGIC_CONTINUE) == 0) return *returnval; /* don't keep searching */ + // So that we print a separator + *printed_something = 0; + firstline = 0; } cont_level = 0; } @@ -460,7 +509,7 @@ varexpand(struct magic_set *ms, char *buf, size_t len, const char *str) size_t l; for (sptr = str; (ptr = strstr(sptr, "${")) != NULL;) { - l = (size_t)(ptr - sptr); + l = CAST(size_t, ptr - sptr); if (l >= len) return -1; memcpy(buf, sptr, l); @@ -526,19 +575,19 @@ mprint(struct magic_set *ms, struct magic *m) switch (m->type) { case FILE_BYTE: - v = file_signextend(ms, m, (uint64_t)p->b); + v = file_signextend(ms, m, CAST(uint64_t, p->b)); switch (check_fmt(ms, desc)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%d", - (unsigned char)v); + CAST(unsigned char, v)); if (file_printf(ms, F(ms, desc, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, desc, "%d"), - (unsigned char) v) == -1) + CAST(unsigned char, v)) == -1) return -1; break; } @@ -548,19 +597,19 @@ mprint(struct magic_set *ms, struct magic *m) case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: - v = file_signextend(ms, m, (uint64_t)p->h); + v = file_signextend(ms, m, CAST(uint64_t, p->h)); switch (check_fmt(ms, desc)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%u", - (unsigned short)v); + CAST(unsigned short, v)); if (file_printf(ms, F(ms, desc, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, desc, "%u"), - (unsigned short) v) == -1) + CAST(unsigned short, v)) == -1) return -1; break; } @@ -571,17 +620,19 @@ mprint(struct magic_set *ms, struct magic *m) case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: - v = file_signextend(ms, m, (uint64_t)p->l); + v = file_signextend(ms, m, CAST(uint64_t, p->l)); switch (check_fmt(ms, desc)) { case -1: return -1; case 1: - (void)snprintf(buf, sizeof(buf), "%u", (uint32_t) v); + (void)snprintf(buf, sizeof(buf), "%u", + CAST(uint32_t, v)); if (file_printf(ms, F(ms, desc, "%s"), buf) == -1) return -1; break; default: - if (file_printf(ms, F(ms, desc, "%u"), (uint32_t) v) == -1) + if (file_printf(ms, F(ms, desc, "%u"), + CAST(uint32_t, v)) == -1) return -1; break; } @@ -597,13 +648,13 @@ mprint(struct magic_set *ms, struct magic *m) return -1; case 1: (void)snprintf(buf, sizeof(buf), "%" INT64_T_FORMAT "u", - (unsigned long long)v); + CAST(unsigned long long, v)); if (file_printf(ms, F(ms, desc, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, desc, "%" INT64_T_FORMAT "u"), - (unsigned long long) v) == -1) + CAST(unsigned long long, v)) == -1) return -1; break; } @@ -615,9 +666,9 @@ mprint(struct magic_set *ms, struct magic *m) case FILE_BESTRING16: case FILE_LESTRING16: if (m->reln == '=' || m->reln == '!') { - if (file_printf(ms, F(ms, desc, "%s"), - file_printable(sbuf, sizeof(sbuf), m->value.s)) - == -1) + if (file_printf(ms, F(ms, desc, "%s"), + file_printable(sbuf, sizeof(sbuf), m->value.s, + sizeof(m->value.s))) == -1) return -1; t = ms->offset + m->vallen; } @@ -632,19 +683,20 @@ mprint(struct magic_set *ms, struct magic *m) if (m->str_flags & STRING_TRIM) { char *last; - while (isspace((unsigned char)*str)) + while (isspace(CAST(unsigned char, *str))) str++; last = str; while (*last) last++; --last; - while (isspace((unsigned char)*last)) + while (isspace(CAST(unsigned char, *last))) last--; *++last = '\0'; } if (file_printf(ms, F(ms, desc, "%s"), - file_printable(sbuf, sizeof(sbuf), str)) == -1) + file_printable(sbuf, sizeof(sbuf), str, + sizeof(p->s) - (str - p->s))) == -1) return -1; if (m->type == FILE_PSTRING) @@ -744,13 +796,14 @@ mprint(struct magic_set *ms, struct magic *m) char *cp; int rval; - cp = strndup((const char *)ms->search.s, ms->search.rm_len); + cp = strndup(RCAST(const char *, ms->search.s), + ms->search.rm_len); if (cp == NULL) { file_oomem(ms, ms->search.rm_len); return -1; } rval = file_printf(ms, F(ms, desc, "%s"), - file_printable(sbuf, sizeof(sbuf), cp)); + file_printable(sbuf, sizeof(sbuf), cp, ms->search.rm_len)); free(cp); if (rval == -1) @@ -776,8 +829,9 @@ mprint(struct magic_set *ms, struct magic *m) t = ms->offset; break; case FILE_DER: - if (file_printf(ms, F(ms, desc, "%s"), - file_printable(sbuf, sizeof(sbuf), ms->ms_value.s)) == -1) + if (file_printf(ms, F(ms, desc, "%s"), + file_printable(sbuf, sizeof(sbuf), ms->ms_value.s, + sizeof(ms->ms_value.s))) == -1) return -1; t = ms->offset; break; @@ -785,7 +839,7 @@ mprint(struct magic_set *ms, struct magic *m) file_magerror(ms, "invalid m->type (%d) in mprint()", m->type); return -1; } - return (int32_t)t; + return CAST(int32_t, t); } private int @@ -832,7 +886,8 @@ moffset(struct magic_set *ms, struct magic *m, const struct buffer *b, p->s[strcspn(p->s, "\r\n")] = '\0'; o = CAST(uint32_t, (ms->offset + strlen(p->s))); if (m->type == FILE_PSTRING) - o += (uint32_t)file_pstring_length_size(m); + o += CAST(uint32_t, + file_pstring_length_size(m)); } break; @@ -898,11 +953,11 @@ moffset(struct magic_set *ms, struct magic *m, const struct buffer *b, case FILE_DER: { o = der_offs(ms, m, nbytes); - if (o == -1 || (size_t)o > nbytes) { + if (o == -1 || CAST(size_t, o) > nbytes) { if ((ms->flags & MAGIC_DEBUG) != 0) { (void)fprintf(stderr, - "Bad DER offset %d nbytes=%zu", - o, nbytes); + "Bad DER offset %d nbytes=%" + SIZE_T_FORMAT "u", o, nbytes); } *op = 0; return 0; @@ -915,10 +970,10 @@ moffset(struct magic_set *ms, struct magic *m, const struct buffer *b, break; } - if ((size_t)o > nbytes) { + if (CAST(size_t, o) > nbytes) { #if 0 - file_error(ms, 0, "Offset out of range %zu > %zu", - (size_t)o, nbytes); + file_error(ms, 0, "Offset out of range %" SIZE_T_FORMAT + "u > %" SIZE_T_FORMAT "u", (size_t)o, nbytes); #endif return -1; } @@ -988,36 +1043,36 @@ cvt_flip(int type, int flip) return type; } } -#define DO_CVT(fld, cast) \ +#define DO_CVT(fld, type) \ if (m->num_mask) \ switch (m->mask_op & FILE_OPS_MASK) { \ case FILE_OPAND: \ - p->fld &= cast m->num_mask; \ + p->fld &= CAST(type, m->num_mask); \ break; \ case FILE_OPOR: \ - p->fld |= cast m->num_mask; \ + p->fld |= CAST(type, m->num_mask); \ break; \ case FILE_OPXOR: \ - p->fld ^= cast m->num_mask; \ + p->fld ^= CAST(type, m->num_mask); \ break; \ case FILE_OPADD: \ - p->fld += cast m->num_mask; \ + p->fld += CAST(type, m->num_mask); \ break; \ case FILE_OPMINUS: \ - p->fld -= cast m->num_mask; \ + p->fld -= CAST(type, m->num_mask); \ break; \ case FILE_OPMULTIPLY: \ - p->fld *= cast m->num_mask; \ + p->fld *= CAST(type, m->num_mask); \ break; \ case FILE_OPDIVIDE: \ - if (cast m->num_mask == 0) \ + if (CAST(type, m->num_mask) == 0) \ return -1; \ - p->fld /= cast m->num_mask; \ + p->fld /= CAST(type, m->num_mask); \ break; \ case FILE_OPMODULO: \ - if (cast m->num_mask == 0) \ + if (CAST(type, m->num_mask) == 0) \ return -1; \ - p->fld %= cast m->num_mask; \ + p->fld %= CAST(type, m->num_mask); \ break; \ } \ if (m->mask_op & FILE_OPINVERSE) \ @@ -1026,61 +1081,61 @@ cvt_flip(int type, int flip) private int cvt_8(union VALUETYPE *p, const struct magic *m) { - DO_CVT(b, (uint8_t)); + DO_CVT(b, uint8_t); return 0; } private int cvt_16(union VALUETYPE *p, const struct magic *m) { - DO_CVT(h, (uint16_t)); + DO_CVT(h, uint16_t); return 0; } private int cvt_32(union VALUETYPE *p, const struct magic *m) { - DO_CVT(l, (uint32_t)); + DO_CVT(l, uint32_t); return 0; } private int cvt_64(union VALUETYPE *p, const struct magic *m) { - DO_CVT(q, (uint64_t)); + DO_CVT(q, uint64_t); return 0; } -#define DO_CVT2(fld, cast) \ +#define DO_CVT2(fld, type) \ if (m->num_mask) \ switch (m->mask_op & FILE_OPS_MASK) { \ case FILE_OPADD: \ - p->fld += cast m->num_mask; \ + p->fld += CAST(type, m->num_mask); \ break; \ case FILE_OPMINUS: \ - p->fld -= cast m->num_mask; \ + p->fld -= CAST(type, m->num_mask); \ break; \ case FILE_OPMULTIPLY: \ - p->fld *= cast m->num_mask; \ + p->fld *= CAST(type, m->num_mask); \ break; \ case FILE_OPDIVIDE: \ - if (cast m->num_mask == 0) \ + if (CAST(type, m->num_mask) == 0) \ return -1; \ - p->fld /= cast m->num_mask; \ + p->fld /= CAST(type, m->num_mask); \ break; \ } \ private int cvt_float(union VALUETYPE *p, const struct magic *m) { - DO_CVT2(f, (float)); + DO_CVT2(f, float); return 0; } private int cvt_double(union VALUETYPE *p, const struct magic *m) { - DO_CVT2(d, (double)); + DO_CVT2(d, double); return 0; } @@ -1136,7 +1191,7 @@ mconvert(struct magic_set *ms, struct magic *m, int flip) * string by p->s, so we need to deduct sz. * Because we can use one of the bytes of the length * after we shifted as NUL termination. - */ + */ len = sz; } while (len--) @@ -1145,14 +1200,14 @@ mconvert(struct magic_set *ms, struct magic *m, int flip) return 1; } case FILE_BESHORT: - p->h = (short)BE16(p); + p->h = CAST(short, BE16(p)); if (cvt_16(p, m) == -1) goto out; return 1; case FILE_BELONG: case FILE_BEDATE: case FILE_BELDATE: - p->l = (int32_t)BE32(p); + p->l = CAST(int32_t, BE32(p)); if (cvt_32(p, m) == -1) goto out; return 1; @@ -1160,19 +1215,19 @@ mconvert(struct magic_set *ms, struct magic *m, int flip) case FILE_BEQDATE: case FILE_BEQLDATE: case FILE_BEQWDATE: - p->q = (uint64_t)BE64(p); + p->q = CAST(uint64_t, BE64(p)); if (cvt_64(p, m) == -1) goto out; return 1; case FILE_LESHORT: - p->h = (short)LE16(p); + p->h = CAST(short, LE16(p)); if (cvt_16(p, m) == -1) goto out; return 1; case FILE_LELONG: case FILE_LEDATE: case FILE_LELDATE: - p->l = (int32_t)LE32(p); + p->l = CAST(int32_t, LE32(p)); if (cvt_32(p, m) == -1) goto out; return 1; @@ -1180,14 +1235,14 @@ mconvert(struct magic_set *ms, struct magic *m, int flip) case FILE_LEQDATE: case FILE_LEQLDATE: case FILE_LEQWDATE: - p->q = (uint64_t)LE64(p); + p->q = CAST(uint64_t, LE64(p)); if (cvt_64(p, m) == -1) goto out; return 1; case FILE_MELONG: case FILE_MEDATE: case FILE_MELDATE: - p->l = (int32_t)ME32(p); + p->l = CAST(int32_t, ME32(p)); if (cvt_32(p, m) == -1) goto out; return 1; @@ -1210,7 +1265,7 @@ mconvert(struct magic_set *ms, struct magic *m, int flip) goto out; return 1; case FILE_BEDOUBLE: - p->q = BE64(p); + p->q = BE64(p); if (cvt_double(p, m) == -1) goto out; return 1; @@ -1301,9 +1356,11 @@ mcopy(struct magic_set *ms, union VALUETYPE *p, int type, int indir, || (b = CAST(const char *, memchr(c, '\r', CAST(size_t, (end - c)))))); lines--, b++) { - last = b; if (b < end - 1 && b[0] == '\r' && b[1] == '\n') b++; + if (b < end - 1 && b[0] == '\n') + b++; + last = b; } if (lines) last = end; @@ -1366,7 +1423,7 @@ mcopy(struct magic_set *ms, union VALUETYPE *p, int type, int indir, * might even cause problems */ if (nbytes < sizeof(*p)) - (void)memset(((char *)(void *)p) + nbytes, '\0', + (void)memset(RCAST(char *, RCAST(void *, p)) + nbytes, '\0', sizeof(*p) - nbytes); return 0; } @@ -1407,7 +1464,7 @@ do_ops(struct magic *m, intmax_t lhs, intmax_t off) if (m->in_op & FILE_OPINVERSE) offset = ~offset; - return (uint32_t)offset; + return CAST(uint32_t, offset); } private int @@ -1428,19 +1485,19 @@ msetoffset(struct magic_set *ms, struct magic *m, struct buffer *bb, return -1; if (o != 0) { // Not yet! - file_magerror(ms, "non zero offset %zu at" - " level %u", o, cont_level); + file_magerror(ms, "non zero offset %" SIZE_T_FORMAT + "u at level %u", o, cont_level); return -1; } - if ((size_t)-m->offset > b->elen) + if (CAST(size_t, -m->offset) > b->elen) return -1; - buffer_init(bb, -1, b->ebuf, b->elen); - ms->eoffset = ms->offset = b->elen + m->offset; + buffer_init(bb, -1, NULL, b->ebuf, b->elen); + ms->eoffset = ms->offset = CAST(int32_t, b->elen + m->offset); } else { if (cont_level == 0) { normal: // XXX: Pass real fd, then who frees bb? - buffer_init(bb, -1, b->fbuf, b->flen); + buffer_init(bb, -1, NULL, b->fbuf, b->flen); ms->offset = m->offset; ms->eoffset = 0; } else { @@ -1448,7 +1505,8 @@ normal: } } if ((ms->flags & MAGIC_DEBUG) != 0) { - fprintf(stderr, "bb=[%p,%zu], %d [b=%p,%zu], [o=%#x, c=%d]\n", + fprintf(stderr, "bb=[%p,%" SIZE_T_FORMAT "u], %d [b=%p,%" + SIZE_T_FORMAT "u], [o=%#x, c=%d]\n", bb->fbuf, bb->flen, ms->offset, b->fbuf, b->flen, m->offset, cont_level); } @@ -1459,7 +1517,8 @@ private int mget(struct magic_set *ms, struct magic *m, const struct buffer *b, const unsigned char *s, size_t nbytes, size_t o, unsigned int cont_level, int mode, int text, int flip, uint16_t *indir_count, uint16_t *name_count, - int *printed_something, int *need_separator, int *returnval) + int *printed_something, int *need_separator, int *returnval, + int *found_match) { uint32_t offset = ms->offset; struct buffer bb; @@ -1484,8 +1543,8 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, - if (mcopy(ms, p, m->type, m->flag & INDIR, s, (uint32_t)(offset + o), - (uint32_t)nbytes, m) == -1) + if (mcopy(ms, p, m->type, m->flag & INDIR, s, + CAST(uint32_t, offset + o), CAST(uint32_t, nbytes), m) == -1) return -1; if ((ms->flags & MAGIC_DEBUG) != 0) { @@ -1494,7 +1553,8 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, "u, il=%hu, nc=%hu)\n", m->type, m->flag, offset, o, nbytes, *indir_count, *name_count); - mdebug(offset, (char *)(void *)p, sizeof(union VALUETYPE)); + mdebug(offset, RCAST(char *, RCAST(void *, p)), + sizeof(union VALUETYPE)); #ifndef COMPILE_ONLY file_mdump(m); #endif @@ -1505,40 +1565,58 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, const int sgn = m->in_op & FILE_OPSIGNED; if (m->in_op & FILE_OPINDIRECT) { const union VALUETYPE *q = CAST(const union VALUETYPE *, - ((const void *)(s + offset + off))); - if (OFFSET_OOB(nbytes, offset + off, sizeof(*q))) - return 0; + RCAST(const void *, s + offset + off)); switch (cvt_flip(m->in_type, flip)) { case FILE_BYTE: + if (OFFSET_OOB(nbytes, offset + off, 1)) + return 0; off = SEXT(sgn,8,q->b); break; case FILE_SHORT: + if (OFFSET_OOB(nbytes, offset + off, 2)) + return 0; off = SEXT(sgn,16,q->h); break; case FILE_BESHORT: + if (OFFSET_OOB(nbytes, offset + off, 2)) + return 0; off = SEXT(sgn,16,BE16(q)); break; case FILE_LESHORT: + if (OFFSET_OOB(nbytes, offset + off, 2)) + return 0; off = SEXT(sgn,16,LE16(q)); break; case FILE_LONG: + if (OFFSET_OOB(nbytes, offset + off, 4)) + return 0; off = SEXT(sgn,32,q->l); break; case FILE_BELONG: case FILE_BEID3: + if (OFFSET_OOB(nbytes, offset + off, 4)) + return 0; off = SEXT(sgn,32,BE32(q)); break; case FILE_LEID3: case FILE_LELONG: + if (OFFSET_OOB(nbytes, offset + off, 4)) + return 0; off = SEXT(sgn,32,LE32(q)); break; case FILE_MELONG: + if (OFFSET_OOB(nbytes, offset + off, 4)) + return 0; off = SEXT(sgn,32,ME32(q)); break; case FILE_BEQUAD: + if (OFFSET_OOB(nbytes, offset + off, 8)) + return 0; off = SEXT(sgn,64,BE64(q)); break; case FILE_LEQUAD: + if (OFFSET_OOB(nbytes, offset + off, 8)) + return 0; off = SEXT(sgn,64,LE64(q)); break; default: @@ -1574,7 +1652,7 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, return 0; lhs = BE32(p); if (in_type == FILE_BEID3) - lhs = cvt_id3(ms, (uint32_t)lhs); + lhs = cvt_id3(ms, CAST(uint32_t, lhs)); offset = do_ops(m, SEXT(sgn,32,lhs), off); break; case FILE_LELONG: @@ -1583,7 +1661,7 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, return 0; lhs = LE32(p); if (in_type == FILE_LEID3) - lhs = cvt_id3(ms, (uint32_t)lhs); + lhs = cvt_id3(ms, CAST(uint32_t, lhs)); offset = do_ops(m, SEXT(sgn,32,lhs), off); break; case FILE_MELONG: @@ -1626,7 +1704,7 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, ms->offset = offset; if ((ms->flags & MAGIC_DEBUG) != 0) { - mdebug(offset, (char *)(void *)p, + mdebug(offset, RCAST(char *, RCAST(void *, p)), sizeof(union VALUETYPE)); #ifndef COMPILE_ONLY file_mdump(m); @@ -1714,7 +1792,8 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, if (rv == 1) { if ((ms->flags & MAGIC_NODESC) == 0 && - file_printf(ms, F(ms, m->desc, "%u"), offset) == -1) { + file_printf(ms, F(ms, m->desc, "%u"), offset) == -1) + { free(rbuf); return -1; } @@ -1744,7 +1823,8 @@ mget(struct magic_set *ms, struct magic *m, const struct buffer *b, *need_separator = 0; rv = match(ms, ml.magic, ml.nmagic, b, offset + o, mode, text, flip, indir_count, name_count, - printed_something, need_separator, returnval); + printed_something, need_separator, returnval, found_match); + (*name_count)--; if (rv != 1) *need_separator = oneed_separator; return rv; @@ -1775,8 +1855,8 @@ file_strncmp(const char *s1, const char *s2, size_t len, uint32_t flags) * the ctype functions will work correctly without extra * casting. */ - const unsigned char *a = (const unsigned char *)s1; - const unsigned char *b = (const unsigned char *)s2; + const unsigned char *a = RCAST(const unsigned char *, s1); + const unsigned char *b = RCAST(const unsigned char *, s2); const unsigned char *eb = b + len; uint64_t v; @@ -1971,13 +2051,15 @@ magiccheck(struct magic_set *ms, struct magic *m) case FILE_STRING: case FILE_PSTRING: l = 0; - v = file_strncmp(m->value.s, p->s, (size_t)m->vallen, m->str_flags); + v = file_strncmp(m->value.s, p->s, CAST(size_t, m->vallen), + m->str_flags); break; case FILE_BESTRING16: case FILE_LESTRING16: l = 0; - v = file_strncmp16(m->value.s, p->s, (size_t)m->vallen, m->str_flags); + v = file_strncmp16(m->value.s, p->s, CAST(size_t, m->vallen), + m->str_flags); break; case FILE_SEARCH: { /* search ms->search.s for the string m->value.s */ @@ -1990,6 +2072,22 @@ magiccheck(struct magic_set *ms, struct magic *m) slen = MIN(m->vallen, sizeof(m->value.s)); l = 0; v = 0; +#ifdef HAVE_MEMMEM + if (slen > 0 && m->str_flags == 0) { + const char *found; + idx = m->str_range + slen; + if (m->str_range == 0 || ms->search.s_len < idx) + idx = ms->search.s_len; + found = CAST(const char *, memmem(ms->search.s, idx, + m->value.s, slen)); + if (!found) + return 0; + idx = found - ms->search.s; + ms->search.offset += idx; + ms->search.rm_len = ms->search.s_len - idx; + break; + } +#endif for (idx = 0; m->str_range == 0 || idx < m->str_range; idx++) { if (slen + idx > ms->search.s_len) @@ -2019,7 +2117,7 @@ magiccheck(struct magic_set *ms, struct magic *m) ((m->str_flags & STRING_IGNORE_CASE) ? REG_ICASE : 0)); if (rc) { file_regerror(&rx, rc, ms); - v = (uint64_t)-1; + v = CAST(uint64_t, -1); } else { regmatch_t pmatch; size_t slen = ms->search.s_len; @@ -2040,15 +2138,15 @@ magiccheck(struct magic_set *ms, struct magic *m) search = CCAST(char *, ""); copy = NULL; } - rc = file_regexec(&rx, (const char *)search, + rc = file_regexec(&rx, RCAST(const char *, search), 1, &pmatch, 0); free(copy); switch (rc) { case 0: - ms->search.s += (int)pmatch.rm_so; - ms->search.offset += (size_t)pmatch.rm_so; - ms->search.rm_len = - (size_t)(pmatch.rm_eo - pmatch.rm_so); + ms->search.s += CAST(int, pmatch.rm_so); + ms->search.offset += CAST(size_t, pmatch.rm_so); + ms->search.rm_len = CAST(size_t, + pmatch.rm_eo - pmatch.rm_so); v = 0; break; @@ -2058,12 +2156,12 @@ magiccheck(struct magic_set *ms, struct magic *m) default: file_regerror(&rx, rc, ms); - v = (uint64_t)-1; + v = CAST(uint64_t, -1); break; } } file_regfree(&rx); - if (v == (uint64_t)-1) + if (v == CAST(uint64_t, -1)) return -1; break; } @@ -2092,7 +2190,7 @@ magiccheck(struct magic_set *ms, struct magic *m) case 'x': if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT - "u == *any* = 1\n", (unsigned long long)v); + "u == *any* = 1\n", CAST(unsigned long long, v)); matched = 1; break; @@ -2100,16 +2198,18 @@ magiccheck(struct magic_set *ms, struct magic *m) matched = v != l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u != %" - INT64_T_FORMAT "u = %d\n", (unsigned long long)v, - (unsigned long long)l, matched); + INT64_T_FORMAT "u = %d\n", + CAST(unsigned long long, v), + CAST(unsigned long long, l), matched); break; case '=': matched = v == l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u == %" - INT64_T_FORMAT "u = %d\n", (unsigned long long)v, - (unsigned long long)l, matched); + INT64_T_FORMAT "u = %d\n", + CAST(unsigned long long, v), + CAST(unsigned long long, l), matched); break; case '>': @@ -2118,15 +2218,16 @@ magiccheck(struct magic_set *ms, struct magic *m) if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u > %" INT64_T_FORMAT "u = %d\n", - (unsigned long long)v, - (unsigned long long)l, matched); + CAST(unsigned long long, v), + CAST(unsigned long long, l), matched); } else { - matched = (int64_t) v > (int64_t) l; + matched = CAST(int64_t, v) > CAST(int64_t, l); if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "d > %" INT64_T_FORMAT "d = %d\n", - (long long)v, (long long)l, matched); + CAST(long long, v), + CAST(long long, l), matched); } break; @@ -2136,15 +2237,16 @@ magiccheck(struct magic_set *ms, struct magic *m) if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u < %" INT64_T_FORMAT "u = %d\n", - (unsigned long long)v, - (unsigned long long)l, matched); + CAST(unsigned long long, v), + CAST(unsigned long long, l), matched); } else { - matched = (int64_t) v < (int64_t) l; + matched = CAST(int64_t, v) < CAST(int64_t, l); if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "d < %" INT64_T_FORMAT "d = %d\n", - (long long)v, (long long)l, matched); + CAST(long long, v), + CAST(long long, l), matched); } break; @@ -2153,8 +2255,9 @@ magiccheck(struct magic_set *ms, struct magic *m) if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "((%" INT64_T_FORMAT "x & %" INT64_T_FORMAT "x) == %" INT64_T_FORMAT - "x) = %d\n", (unsigned long long)v, - (unsigned long long)l, (unsigned long long)l, + "x) = %d\n", CAST(unsigned long long, v), + CAST(unsigned long long, l), + CAST(unsigned long long, l), matched); break; @@ -2163,9 +2266,9 @@ magiccheck(struct magic_set *ms, struct magic *m) if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "((%" INT64_T_FORMAT "x & %" INT64_T_FORMAT "x) != %" INT64_T_FORMAT - "x) = %d\n", (unsigned long long)v, - (unsigned long long)l, (unsigned long long)l, - matched); + "x) = %d\n", CAST(unsigned long long, v), + CAST(unsigned long long, l), + CAST(unsigned long long, l), matched); break; default: @@ -2181,14 +2284,14 @@ private int handle_annotation(struct magic_set *ms, struct magic *m, int firstline) { if ((ms->flags & MAGIC_APPLE) && m->apple[0]) { - if (!firstline && file_printf(ms, "\n- ") == -1) + if (print_sep(ms, firstline) == -1) return -1; if (file_printf(ms, "%.8s", m->apple) == -1) return -1; return 1; } if ((ms->flags & MAGIC_EXTENSION) && m->ext[0]) { - if (!firstline && file_printf(ms, "\n- ") == -1) + if (print_sep(ms, firstline) == -1) return -1; if (file_printf(ms, "%s", m->ext) == -1) return -1; @@ -2197,7 +2300,7 @@ handle_annotation(struct magic_set *ms, struct magic *m, int firstline) if ((ms->flags & MAGIC_MIME_TYPE) && m->mimetype[0]) { char buf[1024]; const char *p; - if (!firstline && file_printf(ms, "\n- ") == -1) + if (print_sep(ms, firstline) == -1) return -1; if (varexpand(ms, buf, sizeof(buf), m->mimetype) == -1) p = m->mimetype; @@ -2213,13 +2316,11 @@ handle_annotation(struct magic_set *ms, struct magic *m, int firstline) private int print_sep(struct magic_set *ms, int firstline) { -// if (ms->flags & MAGIC_NODESC) -// return 0; if (firstline) return 0; /* * we found another match * put a newline and '-' to do some simple formatting */ - return file_printf(ms, "\n- "); + return file_separator(ms); } diff --git a/src/vasprintf.c b/src/vasprintf.c index ad1d3165534d..c87465bd5348 100644 --- a/src/vasprintf.c +++ b/src/vasprintf.c @@ -2,7 +2,7 @@ * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -12,7 +12,7 @@ * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE @@ -51,7 +51,7 @@ form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. The name of the author may not be used to endorse or promote products derived from this software without specific -prior written permission. +prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF @@ -108,7 +108,7 @@ you use strange formats. #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: vasprintf.c,v 1.14 2017/08/13 00:21:47 christos Exp $") +FILE_RCSID("@(#)$File: vasprintf.c,v 1.16 2018/10/01 18:45:39 christos Exp $") #endif /* lint */ #include <assert.h> @@ -116,12 +116,8 @@ FILE_RCSID("@(#)$File: vasprintf.c,v 1.14 2017/08/13 00:21:47 christos Exp $") #include <stdlib.h> #include <stdarg.h> #include <ctype.h> -#ifdef HAVE_LIMITS_H #include <limits.h> -#endif -#ifdef HAVE_STDDEF_H #include <stddef.h> -#endif #define ALLOC_CHUNK 2048 #define ALLOC_SECURITY_MARGIN 1024 /* big value because some platforms have very big 'G' exponent */ diff --git a/tests/CVE-2014-1943.result b/tests/CVE-2014-1943.result new file mode 100644 index 000000000000..68b50898b341 --- /dev/null +++ b/tests/CVE-2014-1943.result @@ -0,0 +1 @@ +Apple Driver Map, blocksize 0
\ No newline at end of file diff --git a/tests/CVE-2014-1943.testfile b/tests/CVE-2014-1943.testfile Binary files differnew file mode 100644 index 000000000000..3fc252bdefb8 --- /dev/null +++ b/tests/CVE-2014-1943.testfile diff --git a/tests/Makefile.am b/tests/Makefile.am index bcb5518d7e70..e84ca13642a2 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -3,18 +3,86 @@ test_LDADD = $(top_builddir)/src/libmagic.la test_CPPFLAGS = -I$(top_builddir)/src EXTRA_DIST = \ -escapevel.result \ +regex-eol.magic \ +CVE-2014-1943.testfile \ +JW07022A.mp3.testfile \ escapevel.testfile \ -gedcom.result \ +fit-map-data.testfile \ gedcom.testfile \ -hddrawcopytool.result \ hddrawcopytool.testfile \ -issue311docx.result \ issue311docx.testfile \ +issue359xlsx.testfile \ +json1.testfile \ +json2.testfile \ +json3.testfile \ +regex-eol.testfile \ +zstd-v0.2-FF.testfile \ +zstd-v0.3-FF.testfile \ +zstd-v0.4-FF.testfile \ +zstd-v0.5-FF.testfile \ +zstd-v0.6-FF.testfile \ +zstd-v0.7-21.testfile \ +zstd-v0.7-22.testfile \ +zstd-v0.8-01.testfile \ +zstd-v0.8-02.testfile \ +zstd-v0.8-03.testfile \ +zstd-v0.8-16.testfile \ +zstd-v0.8-20.testfile \ +zstd-v0.8-21.testfile \ +zstd-v0.8-22.testfile \ +zstd-v0.8-23.testfile \ +zstd-v0.8-F4.testfile \ +zstd-v0.8-FF.testfile \ +CVE-2014-1943.result \ JW07022A.mp3.result \ -JW07022A.mp3.testfile +escapevel.result \ +fit-map-data.result \ +gedcom.result \ +hddrawcopytool.result \ +issue311docx.result \ +issue359xlsx.result \ +json1.result \ +json2.result \ +json3.result \ +regex-eol.result \ +zstd-3-skippable-frames.result \ +zstd-dictionary-0.result \ +zstd-dictionary-1.result \ +zstd-dictionary-2.result \ +zstd-skippable-frame-0.result \ +zstd-skippable-frame-4.result \ +zstd-skippable-frame-8.result \ +zstd-skippable-frame-C.result \ +zstd-v0.2-FF.result \ +zstd-v0.3-FF.result \ +zstd-v0.4-FF.result \ +zstd-v0.5-FF.result \ +zstd-v0.6-FF.result \ +zstd-v0.7-00.result \ +zstd-v0.7-21.result \ +zstd-v0.7-22.result \ +zstd-v0.8-00.result \ +zstd-v0.8-01.result \ +zstd-v0.8-02.result \ +zstd-v0.8-03.result \ +zstd-v0.8-16.result \ +zstd-v0.8-20.result \ +zstd-v0.8-21.result \ +zstd-v0.8-22.result \ +zstd-v0.8-23.result \ +zstd-v0.8-F4.result \ +zstd-v0.8-FF.result T = $(top_srcdir)/tests check-local: MAGIC=$(top_builddir)/magic/magic ./test - set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done + set -e; \ + for i in $T/*.testfile; do \ + echo Running test: $$i; \ + if [ -f $${i%%.testfile}.magic ]; then \ + m=$${i%%.testfile}.magic; \ + else \ + m=$(top_builddir)/magic/magic; \ + fi; \ + TZ=UTC MAGIC=$$m ./test $$i $${i%%.testfile}.result; \ + done diff --git a/tests/Makefile.in b/tests/Makefile.in index aa91243d665a..938a96c864c8 100644 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -292,16 +292,75 @@ top_srcdir = @top_srcdir@ test_LDADD = $(top_builddir)/src/libmagic.la test_CPPFLAGS = -I$(top_builddir)/src EXTRA_DIST = \ -escapevel.result \ +regex-eol.magic \ +CVE-2014-1943.testfile \ +JW07022A.mp3.testfile \ escapevel.testfile \ -gedcom.result \ +fit-map-data.testfile \ gedcom.testfile \ -hddrawcopytool.result \ hddrawcopytool.testfile \ -issue311docx.result \ issue311docx.testfile \ +issue359xlsx.testfile \ +json1.testfile \ +json2.testfile \ +json3.testfile \ +regex-eol.testfile \ +zstd-v0.2-FF.testfile \ +zstd-v0.3-FF.testfile \ +zstd-v0.4-FF.testfile \ +zstd-v0.5-FF.testfile \ +zstd-v0.6-FF.testfile \ +zstd-v0.7-21.testfile \ +zstd-v0.7-22.testfile \ +zstd-v0.8-01.testfile \ +zstd-v0.8-02.testfile \ +zstd-v0.8-03.testfile \ +zstd-v0.8-16.testfile \ +zstd-v0.8-20.testfile \ +zstd-v0.8-21.testfile \ +zstd-v0.8-22.testfile \ +zstd-v0.8-23.testfile \ +zstd-v0.8-F4.testfile \ +zstd-v0.8-FF.testfile \ +CVE-2014-1943.result \ JW07022A.mp3.result \ -JW07022A.mp3.testfile +escapevel.result \ +fit-map-data.result \ +gedcom.result \ +hddrawcopytool.result \ +issue311docx.result \ +issue359xlsx.result \ +json1.result \ +json2.result \ +json3.result \ +regex-eol.result \ +zstd-3-skippable-frames.result \ +zstd-dictionary-0.result \ +zstd-dictionary-1.result \ +zstd-dictionary-2.result \ +zstd-skippable-frame-0.result \ +zstd-skippable-frame-4.result \ +zstd-skippable-frame-8.result \ +zstd-skippable-frame-C.result \ +zstd-v0.2-FF.result \ +zstd-v0.3-FF.result \ +zstd-v0.4-FF.result \ +zstd-v0.5-FF.result \ +zstd-v0.6-FF.result \ +zstd-v0.7-00.result \ +zstd-v0.7-21.result \ +zstd-v0.7-22.result \ +zstd-v0.8-00.result \ +zstd-v0.8-01.result \ +zstd-v0.8-02.result \ +zstd-v0.8-03.result \ +zstd-v0.8-16.result \ +zstd-v0.8-20.result \ +zstd-v0.8-21.result \ +zstd-v0.8-22.result \ +zstd-v0.8-23.result \ +zstd-v0.8-F4.result \ +zstd-v0.8-FF.result T = $(top_srcdir)/tests all: all-am @@ -612,7 +671,16 @@ uninstall-am: check-local: MAGIC=$(top_builddir)/magic/magic ./test - set -e; for i in $T/*.testfile; do echo Running test: $$i; TZ=UTC MAGIC=$(top_builddir)/magic/magic ./test $$i $${i%%.testfile}.result; done + set -e; \ + for i in $T/*.testfile; do \ + echo Running test: $$i; \ + if [ -f $${i%%.testfile}.magic ]; then \ + m=$${i%%.testfile}.magic; \ + else \ + m=$(top_builddir)/magic/magic; \ + fi; \ + TZ=UTC MAGIC=$$m ./test $$i $${i%%.testfile}.result; \ + done # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. diff --git a/tests/fit-map-data.result b/tests/fit-map-data.result new file mode 100644 index 000000000000..0dbbb1b5e356 --- /dev/null +++ b/tests/fit-map-data.result @@ -0,0 +1 @@ +FIT Map data, unit id 65536, serial 3879446968, Sat May 31 10:00:34 2014, manufacturer 1 (garmin), product 1632, type 4 (Activity)
\ No newline at end of file diff --git a/tests/fit-map-data.testfile b/tests/fit-map-data.testfile Binary files differnew file mode 100644 index 000000000000..4f1d46a10b0d --- /dev/null +++ b/tests/fit-map-data.testfile diff --git a/tests/issue359xlsx.result b/tests/issue359xlsx.result new file mode 100644 index 000000000000..d1e15b440767 --- /dev/null +++ b/tests/issue359xlsx.result @@ -0,0 +1 @@ +Microsoft Excel 2007+
\ No newline at end of file diff --git a/tests/issue359xlsx.testfile b/tests/issue359xlsx.testfile Binary files differnew file mode 100644 index 000000000000..9d751c4d0aac --- /dev/null +++ b/tests/issue359xlsx.testfile diff --git a/tests/json1.result b/tests/json1.result new file mode 100644 index 000000000000..01442dc9c271 --- /dev/null +++ b/tests/json1.result @@ -0,0 +1 @@ +JSON data
\ No newline at end of file diff --git a/tests/json1.testfile b/tests/json1.testfile new file mode 100644 index 000000000000..3fb576097051 --- /dev/null +++ b/tests/json1.testfile @@ -0,0 +1,14 @@ + { + "Image": { + "Width": 800, + "Height": 600, + "Title": "View from 15th Floor", + "Thumbnail": { + "Url": "http://www.example.com/image/481989943", + "Height": 125, + "Width": 100 + }, + "Animated" : false, + "IDs": [116, 943, 234, 38793] + } + } diff --git a/tests/json2.result b/tests/json2.result new file mode 100644 index 000000000000..01442dc9c271 --- /dev/null +++ b/tests/json2.result @@ -0,0 +1 @@ +JSON data
\ No newline at end of file diff --git a/tests/json2.testfile b/tests/json2.testfile new file mode 100644 index 000000000000..669007a29d87 --- /dev/null +++ b/tests/json2.testfile @@ -0,0 +1,22 @@ + [ + { + "precision": "zip", + "Latitude": 37.7668, + "Longitude": -122.3959, + "Address": "", + "City": "SAN FRANCISCO", + "State": "CA", + "Zip": "94107", + "Country": "US" + }, + { + "precision": "zip", + "Latitude": 37.371991, + "Longitude": -122.026020, + "Address": "", + "City": "SUNNYVALE", + "State": "CA", + "Zip": "94085", + "Country": "US" + } + ] diff --git a/tests/json3.result b/tests/json3.result new file mode 100644 index 000000000000..01442dc9c271 --- /dev/null +++ b/tests/json3.result @@ -0,0 +1 @@ +JSON data
\ No newline at end of file diff --git a/tests/json3.testfile b/tests/json3.testfile new file mode 100644 index 000000000000..9f31ac1c2d68 --- /dev/null +++ b/tests/json3.testfile @@ -0,0 +1,13 @@ +{ + "abc": "edf", + "json": "crab", + "ololo": [ + 1, + 2, + 3 + ], + "subcrab": { + "name": "crab", + "surname": "subcrab" + } +} diff --git a/tests/regex-eol.magic b/tests/regex-eol.magic new file mode 100644 index 000000000000..883d74c983e6 --- /dev/null +++ b/tests/regex-eol.magic @@ -0,0 +1,6 @@ +## Ansible Vault files +0 string $ANSIBLE_VAULT Ansible Vault text +>&1 regex/1l [0-9]+(\.[0-9]+)+ \b, version %s +>>&1 regex/1l [^;]+$ \b, using %s encryption +!:mime application/ansible-vault +!:strength +60 diff --git a/tests/regex-eol.result b/tests/regex-eol.result new file mode 100644 index 000000000000..5b1a7dbb3f90 --- /dev/null +++ b/tests/regex-eol.result @@ -0,0 +1 @@ +Ansible Vault text, version 1.1, using AES256 encryption
\ No newline at end of file diff --git a/tests/regex-eol.testfile b/tests/regex-eol.testfile new file mode 100644 index 000000000000..607a8524b629 --- /dev/null +++ b/tests/regex-eol.testfile @@ -0,0 +1,24 @@ +$ANSIBLE_VAULT;1.1;AES256 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +00000000000000000000000000000000000000000000000000000000000000000000000000000000 +000000000000000000000000000000000000 diff --git a/tests/zstd-3-skippable-frames.result b/tests/zstd-3-skippable-frames.result new file mode 100644 index 000000000000..468070c34770 --- /dev/null +++ b/tests/zstd-3-skippable-frames.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 1
\ No newline at end of file diff --git a/tests/zstd-dictionary-0.result b/tests/zstd-dictionary-0.result new file mode 100644 index 000000000000..e1b9c620a5e4 --- /dev/null +++ b/tests/zstd-dictionary-0.result @@ -0,0 +1 @@ +Zstandard dictionary (ID 0)
\ No newline at end of file diff --git a/tests/zstd-dictionary-1.result b/tests/zstd-dictionary-1.result new file mode 100644 index 000000000000..dd9f5f1b1e50 --- /dev/null +++ b/tests/zstd-dictionary-1.result @@ -0,0 +1 @@ +Zstandard dictionary (ID 1)
\ No newline at end of file diff --git a/tests/zstd-dictionary-2.result b/tests/zstd-dictionary-2.result new file mode 100644 index 000000000000..e1ac5b3a9d97 --- /dev/null +++ b/tests/zstd-dictionary-2.result @@ -0,0 +1 @@ +Zstandard dictionary (ID 285212672)
\ No newline at end of file diff --git a/tests/zstd-skippable-frame-0.result b/tests/zstd-skippable-frame-0.result new file mode 100644 index 000000000000..a4e3e482ce6b --- /dev/null +++ b/tests/zstd-skippable-frame-0.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.2)
\ No newline at end of file diff --git a/tests/zstd-skippable-frame-4.result b/tests/zstd-skippable-frame-4.result new file mode 100644 index 000000000000..cecb5e71a5e6 --- /dev/null +++ b/tests/zstd-skippable-frame-4.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.3)
\ No newline at end of file diff --git a/tests/zstd-skippable-frame-8.result b/tests/zstd-skippable-frame-8.result new file mode 100644 index 000000000000..528f701d059e --- /dev/null +++ b/tests/zstd-skippable-frame-8.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.4)
\ No newline at end of file diff --git a/tests/zstd-skippable-frame-C.result b/tests/zstd-skippable-frame-C.result new file mode 100644 index 000000000000..468070c34770 --- /dev/null +++ b/tests/zstd-skippable-frame-C.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 1
\ No newline at end of file diff --git a/tests/zstd-v0.2-FF.result b/tests/zstd-v0.2-FF.result new file mode 100644 index 000000000000..a4e3e482ce6b --- /dev/null +++ b/tests/zstd-v0.2-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.2)
\ No newline at end of file diff --git a/tests/zstd-v0.2-FF.testfile b/tests/zstd-v0.2-FF.testfile new file mode 100644 index 000000000000..6fe4f2737558 --- /dev/null +++ b/tests/zstd-v0.2-FF.testfile @@ -0,0 +1 @@ +"µ/ýÿ
\ No newline at end of file diff --git a/tests/zstd-v0.3-FF.result b/tests/zstd-v0.3-FF.result new file mode 100644 index 000000000000..cecb5e71a5e6 --- /dev/null +++ b/tests/zstd-v0.3-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.3)
\ No newline at end of file diff --git a/tests/zstd-v0.3-FF.testfile b/tests/zstd-v0.3-FF.testfile new file mode 100644 index 000000000000..dc5046911fe8 --- /dev/null +++ b/tests/zstd-v0.3-FF.testfile @@ -0,0 +1 @@ +#µ/ýÿ
\ No newline at end of file diff --git a/tests/zstd-v0.4-FF.result b/tests/zstd-v0.4-FF.result new file mode 100644 index 000000000000..528f701d059e --- /dev/null +++ b/tests/zstd-v0.4-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.4)
\ No newline at end of file diff --git a/tests/zstd-v0.4-FF.testfile b/tests/zstd-v0.4-FF.testfile new file mode 100644 index 000000000000..f2768a21a8c0 --- /dev/null +++ b/tests/zstd-v0.4-FF.testfile @@ -0,0 +1 @@ +$µ/ýÿ
\ No newline at end of file diff --git a/tests/zstd-v0.5-FF.result b/tests/zstd-v0.5-FF.result new file mode 100644 index 000000000000..815c80d39a33 --- /dev/null +++ b/tests/zstd-v0.5-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.5)
\ No newline at end of file diff --git a/tests/zstd-v0.5-FF.testfile b/tests/zstd-v0.5-FF.testfile new file mode 100644 index 000000000000..a25f337be50e --- /dev/null +++ b/tests/zstd-v0.5-FF.testfile @@ -0,0 +1 @@ +%µ/ýÿ
\ No newline at end of file diff --git a/tests/zstd-v0.6-FF.result b/tests/zstd-v0.6-FF.result new file mode 100644 index 000000000000..b28af6b21947 --- /dev/null +++ b/tests/zstd-v0.6-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.6)
\ No newline at end of file diff --git a/tests/zstd-v0.6-FF.testfile b/tests/zstd-v0.6-FF.testfile new file mode 100644 index 000000000000..1c8ca59847ec --- /dev/null +++ b/tests/zstd-v0.6-FF.testfile @@ -0,0 +1 @@ +&µ/ýÿ
\ No newline at end of file diff --git a/tests/zstd-v0.7-00.result b/tests/zstd-v0.7-00.result new file mode 100644 index 000000000000..72e0e27aa907 --- /dev/null +++ b/tests/zstd-v0.7-00.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.7), Dictionary ID: None
\ No newline at end of file diff --git a/tests/zstd-v0.7-21.result b/tests/zstd-v0.7-21.result new file mode 100644 index 000000000000..a4d43c819f27 --- /dev/null +++ b/tests/zstd-v0.7-21.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.7), Dictionary ID: 1
\ No newline at end of file diff --git a/tests/zstd-v0.7-21.testfile b/tests/zstd-v0.7-21.testfile new file mode 100644 index 000000000000..b40294ea5ad1 --- /dev/null +++ b/tests/zstd-v0.7-21.testfile @@ -0,0 +1 @@ +'µ/ý!
\ No newline at end of file diff --git a/tests/zstd-v0.7-22.result b/tests/zstd-v0.7-22.result new file mode 100644 index 000000000000..b4336c9a3990 --- /dev/null +++ b/tests/zstd-v0.7-22.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.7), Dictionary ID: 513
\ No newline at end of file diff --git a/tests/zstd-v0.7-22.testfile b/tests/zstd-v0.7-22.testfile new file mode 100644 index 000000000000..8b72d683256b --- /dev/null +++ b/tests/zstd-v0.7-22.testfile @@ -0,0 +1 @@ +'µ/ý"
\ No newline at end of file diff --git a/tests/zstd-v0.8-00.result b/tests/zstd-v0.8-00.result new file mode 100644 index 000000000000..67edebd88cd2 --- /dev/null +++ b/tests/zstd-v0.8-00.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: None
\ No newline at end of file diff --git a/tests/zstd-v0.8-01.result b/tests/zstd-v0.8-01.result new file mode 100644 index 000000000000..a1239a71f56a --- /dev/null +++ b/tests/zstd-v0.8-01.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 2
\ No newline at end of file diff --git a/tests/zstd-v0.8-01.testfile b/tests/zstd-v0.8-01.testfile new file mode 100644 index 000000000000..88735e4704b5 --- /dev/null +++ b/tests/zstd-v0.8-01.testfile @@ -0,0 +1 @@ +(µ/ý
\ No newline at end of file diff --git a/tests/zstd-v0.8-02.result b/tests/zstd-v0.8-02.result new file mode 100644 index 000000000000..82beab8ae14e --- /dev/null +++ b/tests/zstd-v0.8-02.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 770
\ No newline at end of file diff --git a/tests/zstd-v0.8-02.testfile b/tests/zstd-v0.8-02.testfile new file mode 100644 index 000000000000..db554336c7ce --- /dev/null +++ b/tests/zstd-v0.8-02.testfile @@ -0,0 +1 @@ +(µ/ý
\ No newline at end of file diff --git a/tests/zstd-v0.8-03.result b/tests/zstd-v0.8-03.result new file mode 100644 index 000000000000..6aaaa197e805 --- /dev/null +++ b/tests/zstd-v0.8-03.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 84148994
\ No newline at end of file diff --git a/tests/zstd-v0.8-03.testfile b/tests/zstd-v0.8-03.testfile new file mode 100644 index 000000000000..506b344ac901 --- /dev/null +++ b/tests/zstd-v0.8-03.testfile @@ -0,0 +1 @@ +(µ/ý
\ No newline at end of file diff --git a/tests/zstd-v0.8-16.result b/tests/zstd-v0.8-16.result new file mode 100644 index 000000000000..82beab8ae14e --- /dev/null +++ b/tests/zstd-v0.8-16.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 770
\ No newline at end of file diff --git a/tests/zstd-v0.8-16.testfile b/tests/zstd-v0.8-16.testfile new file mode 100644 index 000000000000..3f87f79daf81 --- /dev/null +++ b/tests/zstd-v0.8-16.testfile @@ -0,0 +1 @@ +(µ/ý
\ No newline at end of file diff --git a/tests/zstd-v0.8-20.result b/tests/zstd-v0.8-20.result new file mode 100644 index 000000000000..67edebd88cd2 --- /dev/null +++ b/tests/zstd-v0.8-20.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: None
\ No newline at end of file diff --git a/tests/zstd-v0.8-20.testfile b/tests/zstd-v0.8-20.testfile new file mode 100644 index 000000000000..76fdbb8ac190 --- /dev/null +++ b/tests/zstd-v0.8-20.testfile @@ -0,0 +1 @@ +(µ/ý
\ No newline at end of file diff --git a/tests/zstd-v0.8-21.result b/tests/zstd-v0.8-21.result new file mode 100644 index 000000000000..468070c34770 --- /dev/null +++ b/tests/zstd-v0.8-21.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 1
\ No newline at end of file diff --git a/tests/zstd-v0.8-21.testfile b/tests/zstd-v0.8-21.testfile new file mode 100644 index 000000000000..9ebeff4833f1 --- /dev/null +++ b/tests/zstd-v0.8-21.testfile @@ -0,0 +1 @@ +(µ/ý!
\ No newline at end of file diff --git a/tests/zstd-v0.8-22.result b/tests/zstd-v0.8-22.result new file mode 100644 index 000000000000..9869d24f2671 --- /dev/null +++ b/tests/zstd-v0.8-22.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 513
\ No newline at end of file diff --git a/tests/zstd-v0.8-22.testfile b/tests/zstd-v0.8-22.testfile new file mode 100644 index 000000000000..f2e55bf77f00 --- /dev/null +++ b/tests/zstd-v0.8-22.testfile @@ -0,0 +1 @@ +(µ/ý"
\ No newline at end of file diff --git a/tests/zstd-v0.8-23.result b/tests/zstd-v0.8-23.result new file mode 100644 index 000000000000..d38420fed117 --- /dev/null +++ b/tests/zstd-v0.8-23.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 67305985
\ No newline at end of file diff --git a/tests/zstd-v0.8-23.testfile b/tests/zstd-v0.8-23.testfile new file mode 100644 index 000000000000..f66a18fc2467 --- /dev/null +++ b/tests/zstd-v0.8-23.testfile @@ -0,0 +1 @@ +(µ/ý#
\ No newline at end of file diff --git a/tests/zstd-v0.8-F4.result b/tests/zstd-v0.8-F4.result new file mode 100644 index 000000000000..67edebd88cd2 --- /dev/null +++ b/tests/zstd-v0.8-F4.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: None
\ No newline at end of file diff --git a/tests/zstd-v0.8-F4.testfile b/tests/zstd-v0.8-F4.testfile new file mode 100644 index 000000000000..a4e4240ec80e --- /dev/null +++ b/tests/zstd-v0.8-F4.testfile @@ -0,0 +1 @@ +(µ/ýô
\ No newline at end of file diff --git a/tests/zstd-v0.8-FF.result b/tests/zstd-v0.8-FF.result new file mode 100644 index 000000000000..d38420fed117 --- /dev/null +++ b/tests/zstd-v0.8-FF.result @@ -0,0 +1 @@ +Zstandard compressed data (v0.8+), Dictionary ID: 67305985
\ No newline at end of file diff --git a/tests/zstd-v0.8-FF.testfile b/tests/zstd-v0.8-FF.testfile new file mode 100644 index 000000000000..bc639113edaf --- /dev/null +++ b/tests/zstd-v0.8-FF.testfile @@ -0,0 +1 @@ +(µ/ýÿ
\ No newline at end of file |