reducing dependency on shell escape

[davidcarlisle]

This is a feature request rather than a bug report.

I am trying to get minted to run in the default restricted shell escape mode of texlive.

I have a latex server (see https://www.learnlatex.org/) returning pdfs generated from user supplied tex source.
It would be nice to be able to include minted examples...

It's running in a fairly sandboxed environment and I'd be prepared to allow pygmentize (which is installed) but not unrestricted shell escape.

So as an experiment I added pygmentize to the list of allowed commands on texmf.cnf

shell_escape_commands = \
bibtex,bibtex8,\
extractbb,\
gregorio,\
kpsewhich,\
makeindex,\
repstopdf,\
r-mpost,\
texosquery-jre8,\
pygmentize,\

Unfortunately this doesn't work and minted wants access to \ShellEscape{rm and a host of other things that can't be added to the list of allowed shell commands.

In such a context I don't need minted to clean up after itself as the entire working directory tree will be deleted the moment the job finishes, so I probably(?) don't need it to have access to rm.

I started to trace what would need to be changed but got a bit lost, does this sound feasible or are there too many essential shell operations the code relies on?

[muzimuzhi]

It seems that in restricted write18, redirection > is not supported.

With pygmentize appended to shell_escape_commands, the following example doesn't work:

\documentclass{article}
\usepackage{shellesc}

\begin{document}
\ShellEscape{pygmentize -S default -f latex -P commandprefix=PYGdefault > ./default.pygstyle}
\end{document}

This can be further simplified to \ShellEscape{echo "abc" > abc.txt}, with echo appended to shell_escape_commands.

From the doc of pygments, it uses stdout when option -S is used

With the -S option, print out style definitions for style <style>
for formatter <formatter>. The argument given by -a is formatter
dependent.

Of course, one can write a wrapper python script to support syntax of

pygmentize -S default -f latex -P commandprefix=PYGdefault -o default.pygstyle

[gpoore]

All the pygmentize commands could be switched to use -o <outfile> to eliminate the redirect issue @muzimuzhi pointed out.

If cleanup isn't an issue, then \DeleteFile isn't needed, which removes the rm usage. That leaves \ProvideDirectory and \TestAppExists as the other typical uses of \ShellEscape. \ProvideDirectory could be avoided by using a hardcoded cache dir, or can be skipped altogether if caching is turned off. \TestAppExists can be disabled if you know pygmentize is installed.

So actually using -o <outfile> plus no cache plus \def\DeleteFile#1{} and \def\TestAppExists#1{} should cover most cases. The other (rarer) case is the autogobble option that requires running Python. Some or all of this may be worth incorporating into minted as a package option. It might also be possible to handle some of this by having the server always include some sort of patching package that does a few \AtEndPreamble etc. checks and patches.

[davidcarlisle]

Thanks. I'd started adjusting \DeleteFile, \TestAppExist and friends but missed the redirect, from the comments it seems that it should be feasible; I will probably try again and report back here but probably not this week.

[muzimuzhi]

All the pygmentize commands could be switched to use -o <outfile> to eliminate the redirect issue @muzimuzhi pointed out.

@gpoore
Can you give some hints? I checked the implementation of -S option in pygments, it directly calls print().
https://github.com/pygments/pygments/blob/a486735f2b485ed35bcb60101fc9a3e2a0eca11d/pygments/cmdline.py#L295-L315

@davidcarlisle
Here (https://github.com/muzimuzhi/minted/tree/reduce-shell-escape) is a first stage implementation. It adds a new pkg option "reduceshellescape", which now just sets cache=. and disables \DeleteFile, TestAppExist, \ProvideDirectory, and the check for \pdf@shellescape. Apart from the redirection issue, it works well.

[gpoore]

@muzimuzhi You're right about -S...I think I misread the pygmentize help. In that case, getting this to work would probably involve creating a special wrapper script for pygmentize to create the styles. The wrapper script could be on the list of allowed commands and would basically just call pygmentize but capture its stdout and save to a file. Another, possibly simpler, option would be to just pregenerate all styles, so the style-generating code is never called. That should be possible in a server context like this.

[davidcarlisle]

On my own server I can add pygmentize to the restricted shell list (in fact I have done so) but if I were to try to get it added by default so minted could be used by normal setup without shell escape then I'd have to answer this question which has been in the default config for years

shell_escape_commands = \
bibtex,bibtex8,\
extractbb,\
gregorio,\
kpsewhich,\
makeindex,\
repstopdf,\
r-mpost,\
texosquery-jre8,\

% we'd like to allow:
% dvips - but external commands can be executed, need at least -R1.
% epspdf, ps2pdf, pstopdf - need to respect openout_any,
%   and gs -dSAFER must be used and check for shell injection with filenames.
% pygmentize - but is the filter feature insecure?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So if we were to have a wrapper script that added -o perhaps it could also simply disable some of the options, I note even pygmentize --help warns about x option

Users should be very careful not to use this option with untrusted files, because it will import and run them.

so pygmentize would not be allowed in the default texlive list as is. This wouldn't be unusual: epstopdf and mpost have r wrappers with restricted functionality that are included here for the same reason.

@gpoore wrote

it might also be possible to handle some of this by having the server always include some sort of patching package that does a few \AtEndPreamble etc. checks and patches.

Yes I already do some of that (for other packages) although I try to avoid it too much as applying regex replace to incoming documents has a potential to mess things up and confuse the user greatly:-)

But while my immediate concern is a server setup I think this would be useful in general. minted is often the answer to code listing questions on sites like https://tex.stackexchange.com but it always concerns me when users are told "run this obscure fragment of tex code and use an option that gives it write access to all your files" in practice nothing bad happens and code gets syntax highlighted but the community generally seems to have a lack of awareness about why shell escape is disabled by default. If most uses of minted could be run in the default mode I think it would be a good thing.

[gpoore]

@davidcarlisle Thanks for pointing out the reason pygmentize isn't on the restricted shell list! I've actually been thinking about a wrapper for pygmentize to add some additional functionality (mostly simple things like autogobble that are trivial in Python but difficult in LaTeX--this currently involves its own \ShellEsc, increasing complexity and overhead). If a wrapper could also eliminate the need for --shell-escape, that would be really great!

[muzimuzhi]

(a little off-topic) One concern, will texlive accept to add a new command to shell_escape_commands which is not distributed with texlive itself and must be manually installed?

[davidcarlisle]

@muzimuzhi well I can ask Karl:-) epstopdf relies on gs and (on most platforms) that isn't part of texlive but needs to be in installed in the system. texosquery is in the list and needs java, so I don't think it is impossible to get this added if it is sufficiently restricted.

[gpoore]

Update: minted version 3.0 is now under development, thanks to a grant from the TeX Users Group. It will replace pygmentize with a new Python executable that is designed to be compatible with restricted shell escape. Initial beta releases of minted version 3.0 are expected by early 2024.

[gpoore]

minted v3.0.0beta1, which is designed to be compatible with restricted shell escape, is now available. If you would like to experiment with the beta version, installation instructions are here and the CHANGELOG includes details about new features (the PDF documentation won't be fully updated until the final v3 release). Otherwise, the final v3 release should be available sometime in June.

[samcarter]

I'm very excited about these developments, so I tried to experiment with v3. I tried the following test file:

\documentclass{article}

\usepackage{minted}

\begin{document}
test
\end{document}

However I get the following error:

This is pdfTeX, Version 3.141592653-2.6-1.40.26 (TeX Live 2024) (preloaded format=pdflatex 2024.5.23)  25 MAY 2024 21:25
entering extended mode
 restricted \write18 enabled.
 %&-line parsing enabled.
**document.tex
(./document.tex
LaTeX2e <2023-11-01> patch level 1
L3 programming layer <2024-05-08>
(/usr/local/texlive/2024/texmf-dist/tex/latex/base/article.cls
Document Class: article 2023/05/17 v1.4n Standard LaTeX document class
(/usr/local/texlive/2024/texmf-dist/tex/latex/base/size10.clo
File: size10.clo 2023/05/17 v1.4n Standard LaTeX file (size option)
)
\c@part=\count189
\c@section=\count190
\c@subsection=\count191
\c@subsubsection=\count192
\c@paragraph=\count193
\c@subparagraph=\count194
\c@figure=\count195
\c@table=\count196
\abovecaptionskip=\skip48
\belowcaptionskip=\skip49
\bibindent=\dimen140
) (./minted.sty
Package: minted 2024/05/25 v3.0.0beta1 Yet another Pygments shim for LaTeX

(/usr/local/texlive/2024/texmf-dist/tex/generic/catchfile/catchfile.sty
Package: catchfile 2019/12/09 v1.8 Catch the contents of a file (HO)

(/usr/local/texlive/2024/texmf-dist/tex/generic/infwarerr/infwarerr.sty
Package: infwarerr 2019/12/03 v1.5 Providing info/warning/error messages (HO)
)
(/usr/local/texlive/2024/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty
Package: ltxcmds 2023-12-04 v1.26 LaTeX kernel commands for general use (HO)
)
(/usr/local/texlive/2024/texmf-dist/tex/generic/etexcmds/etexcmds.sty
Package: etexcmds 2019/12/15 v1.7 Avoid name clashes with e-TeX commands (HO)

(/usr/local/texlive/2024/texmf-dist/tex/generic/iftex/iftex.sty
Package: iftex 2022/02/03 v1.0f TeX engine tests
)))
(/usr/local/texlive/2024/texmf-dist/tex/latex/etoolbox/etoolbox.sty
Package: etoolbox 2020/10/05 v2.5k e-TeX tools for LaTeX (JAW)
\etb@tempcnta=\count197
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/framed/framed.sty
Package: framed 2011/10/22 v 0.96: framed or shaded text with page breaks
\OuterFrameSep=\skip50
\fb@frw=\dimen141
\fb@frh=\dimen142
\FrameRule=\dimen143
\FrameSep=\dimen144
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/fvextra/fvextra.sty
Package: fvextra 2024/05/16 v1.7.0 fvextra - extensions and patches for fancyvr
b

(/usr/local/texlive/2024/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty
Package: fancyvrb 2024/01/20 4.5c verbatim text (tvz,hv)

(/usr/local/texlive/2024/texmf-dist/tex/latex/graphics/keyval.sty
Package: keyval 2022/05/29 v1.15 key=value parser (DPC)
\KV@toks@=\toks17
)
\FV@CodeLineNo=\count198
\FV@InFile=\read2
\FV@TabBox=\box51
\c@FancyVerbLine=\count199
\FV@StepNumber=\count266
\FV@OutFile=\write3
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/upquote/upquote.sty
Package: upquote 2012/04/19 v1.3 upright-quote and grave-accent glyphs in verba
tim
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/lineno/lineno.sty
Package: lineno 2023/05/20 line numbers on paragraphs v5.3

(/usr/local/texlive/2024/texmf-dist/tex/latex/kvoptions/kvoptions.sty
Package: kvoptions 2022-06-15 v3.15 Key value format for package options (HO)

(/usr/local/texlive/2024/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty
Package: kvsetkeys 2022-10-05 v1.19 Key value parser (HO)
))
\linenopenalty=\count267
\output=\toks18
\linenoprevgraf=\count268
\linenumbersep=\dimen145
\linenumberwidth=\dimen146
\c@linenumber=\count269
\c@pagewiselinenumber=\count270
\c@LN@truepage=\count271
\c@internallinenumber=\count272
\c@internallinenumbers=\count273
\quotelinenumbersep=\dimen147
\bframerule=\dimen148
\bframesep=\dimen149
\bframebox=\box52
LaTeX Info: Redefining \\ on input line 3180.
)
\c@FancyVerbWriteLine=\count274
\c@FancyVerbBufferIndex=\count275
\c@FancyVerbBufferLength=\count276
\c@FancyVerbBufferLine=\count277
\c@FV@oldbufferlength=\count278
\c@FV@TrueTabGroupLevel=\count279
\c@FV@TrueTabCounter=\count280
\FV@TabBox@Group=\box53
\FV@TmpLength=\skip51
\c@FV@HighlightLinesStart=\count281
\c@FV@HighlightLinesStop=\count282
\FV@LoopCount=\count283
\FV@NCharsBox=\box54
\FV@BreakIndent=\dimen150
\FV@BreakIndentNChars=\count284
\FV@BreakSymbolSepLeft=\dimen151
\FV@BreakSymbolSepLeftNChars=\count285
\FV@BreakSymbolSepRight=\dimen152
\FV@BreakSymbolSepRightNChars=\count286
\FV@BreakSymbolIndentLeft=\dimen153
\FV@BreakSymbolIndentLeftNChars=\count287
\FV@BreakSymbolIndentRight=\dimen154
\FV@BreakSymbolIndentRightNChars=\count288
\c@FancyVerbLineBreakLast=\count289
\FV@LineBox=\box55
\FV@LineIndentBox=\box56
\c@FV@BreakBufferDepth=\count290
\FV@LineWidth=\dimen155
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/latex2pydata/latex2pydata.sty
Package: latex2pydata 2024/05/16 v0.2.0 latex2pydata - write data to file in Py
thon literal format

(/usr/local/texlive/2024/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
Package: pdftexcmds 2020-06-27 v0.33 Utility functions of pdfTeX for LuaTeX (HO
)
Package pdftexcmds Info: \pdf@primitive is available.
Package pdftexcmds Info: \pdf@ifprimitive is available.
Package pdftexcmds Info: \pdfdraftmode found.
)
\c@pydata@bufferindex=\count291
\c@pydata@defaultbufferlength=\count292
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/local/texlive/2024/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
\pgfkeys@pathtoks=\toks19
\pgfkeys@temptoks=\toks20

(/usr/local/texlive/2024/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfil
tered.code.tex
\pgfkeys@tmptoks=\toks21
)))
(/usr/local/texlive/2024/texmf-dist/tex/latex/pgfopts/pgfopts.sty
Package: pgfopts 2014/07/10 v2.1a LaTeX package options with pgfkeys
\pgfopts@list@add@a@toks=\toks22
\pgfopts@list@add@b@toks=\toks23
)
(/usr/local/texlive/2024/texmf-dist/tex/latex/tools/shellesc.sty
Package: shellesc 2023/07/08 v1.0d unified shell escape interface for LaTeX
Package shellesc Info: Restricted shell escape enabled on input line 77.
)
! Argument of \minted@creationdatetotimestamp@i has an extra }.
<inserted text> 
                \par 
l.79 ...creationdatetotimestamp{\pdfcreationdate}}
                                                  
I've run across a `}' that doesn't seem to match anything.
For example, `\def\a#1{...}' and `\a}' would produce
this error. If you simply proceed now, the `\par' that
I've just inserted will cause me to report a runaway
argument that might be the root of the problem. But if
your `}' was spurious, just type `2' and it will go away.

Runaway argument?
20240525212534+02'00'\relax 
! Paragraph ended before \minted@creationdatetotimestamp@i was complete.
<to be read again> 
                   \par 
l.79 ...creationdatetotimestamp{\pdfcreationdate}}
                                                  
I suspect you've forgotten a `}', causing me to apply this
control sequence to too much text. How can we recover?
My plan is to forget the whole thing and hope for the best.

Any idea what I'm doing wrong?

My latexminted version is 0.1.0b1 (libraries: latex2pydata 0.2.0, pygments 2.18.0) (just installed via pip)

[gpoore]

@samcarter It should work now with 36a0a5f. There was a bug in timestamp parsing.

[samcarter]

@gpoore Thanks a lot! This solves the problem!

[romildo]

I have tried the following example with minted V3 and -output-dir, but it fails.

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{minted}

\begin{document}

Here are some small tests for the \texttt{minted} package.

\begin{minted}[mathescape,
               linenos,
               numbersep=5pt,
               gobble=2,
               frame=lines,
               framesep=2mm]{csharp}
string title = "This is a Unicode π in the sky"
/*
Defined as $\pi=\lim_{n\to\infty}\frac{P_n}{d}$ where $P$ is the perimeter
of an $n$-sided regular polygon circumscribing a
circle of diameter $d$.
*/
const double pi = 3.1415926535
\end{minted}

\end{document}

$ mkdir _build

$ pdflatex -output-directory=_build --shell-escape minted-test
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023/nixos.org) (preloaded format=pdflatex)
 \write18 enabled.
entering extended mode
(./minted-test.tex
LaTeX2e <2023-11-01> patch level 1
L3 programming layer <2024-02-20>

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/article.cls
Document Class: article 2023/05/17 v1.4n Standard LaTeX document class

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/size10.clo))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/inputenc.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/fontenc.sty) (/home/romildo/texmf/tex/latex/minted/minted.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/catchfile/catchfile.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/infwarerr/infwarerr.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/ltxcmds/ltxcmds.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/etexcmds/etexcmds.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/iftex/iftex.sty)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/etoolbox/etoolbox.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/framed/framed.sty) (/home/romildo/texmf/tex/latex/fvextra/fvextra.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/fancyvrb/fancyvrb.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics/keyval.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/upquote/upquote.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/textcomp.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/lineno/lineno.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvoptions/kvoptions.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvsetkeys/kvsetkeys.sty))))
(/home/romildo/texmf/tex/latex/latex2pydata/latex2pydata.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pdftexcmds/pdftexcmds.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgf/utilities/pgfkeys.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeys.code.tex
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeyslibraryfiltered.code.tex)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgfopts/pgfopts.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/tools/shellesc.sty)system returned with code 256


Package minted Warning: minted v3+ Python executable is not installed or is not
 added to PATH; falling back to minted v2 (install executable, or switch to pac
kage minted2) on input line 215.

(/home/romildo/texmf/tex/latex/minted/minted2.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/ifthen.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/ifplatform/ifplatform.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/iftex/ifluatex.sty) (./minted-test.w18))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/xstring/xstring.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/xstring/xstring.tex))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/float/float.sty))rm: cannot remove '_3F29AA7400DFA670869ED2F056F07532.data.minted': No such file or directory
system returned with code 256
)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/xcolor/xcolor.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics-cfg/color.cfg)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics-def/pdftex.def)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics/mathcolor.ltx))/run/current-system/sw/bin/pygmentize

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/l3backend/l3backend-pdftex.def)
No file minted-test.aux.

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/t1cmtt.fd) (./_minted-minted-test/default.pygstyle)Error: cannot read infile: [Errno 2] No such file or directory: 'minted-test.pyg'
system returned with code 256


! Package minted Error: Missing Pygments output; \inputminted was
probably given a file that does not exist--otherwise, you may need
the outputdir package option, or may be using an incompatible build tool,
or may be using frozencache with a missing file.

See the minted package documentation for explanation.
Type  H <return>  for immediate help.
 ...

l.24 \end{minted}

?
! Emergency stop.
 ...

l.24 \end{minted}

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on _build/minted-test.log.

$ ls -AlF
total 24
-rw-r--r-- 1 romildo romildo  768 mai 26 17:20 _3F29AA7400DFA670869ED2F056F07532.errlog.minted
-rw-r--r-- 1 romildo romildo  158 mai 26 17:20 _3F29AA7400DFA670869ED2F056F07532.message.minted
drwxr-xr-x 2 romildo romildo 4096 mai 26 17:20 _build/
drwxr-xr-x 2 romildo romildo 4096 mai 26 17:20 _minted-minted-test/
-rw-r--r-- 1 romildo romildo  611 mai 26 17:14 minted-test.tex
-rw-r--r-- 1 romildo romildo   94 mai 26 10:03 test.tex

$ ls -AlF _build
total 28
-rw-r--r-- 1 romildo romildo    98 mai 26 17:20 _3F29AA7400DFA670869ED2F056F07532.data.minted
-rw-r--r-- 1 romildo romildo     8 mai 26 17:20 minted-test.aux
-rw-r--r-- 1 romildo romildo 14402 mai 26 17:20 minted-test.log
-rw-r--r-- 1 romildo romildo   234 mai 26 17:20 minted-test.pyg

$ ls -AlF _minted-minted-test/
total 8
-rw-r--r-- 1 romildo romildo 6428 mai 26 17:20 default.pygstyle

[gpoore]

@romildo There's a warning message that the minted v3 Python executable can't be found, so it is falling back to v2 using the new minted2 package.

[romildo]

There's a warning message that the minted v3 Python executable can't be found, so it is falling back to v2 using the new minted2 package.

By minted v3 Python executable you mean the program latexminted, right?

If so, it is available at the command line:

$ type latexminted
latexminted is /run/current-system/sw/bin/latexminted

$ echo $PATH
/home/romildo/.local/bin:/run/wrappers/bin:/home/romildo/.nix-profile/bin:/nix/profile/bin:/home/romildo/.local/state/nix/profile/bin:/etc/profiles/per-user/romildo/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin

$ latexminted --version
latexminted 0.1.0b1 (libraries: latex2pydata 0.2.0, pygments 2.17.2)

$ pdflatex --shell-escape minted-test
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023/nixos.org) (preloaded format=pdflatex)
[...]
Package minted Warning: minted v3+ Python executable is not installed or is not
 added to PATH; falling back to minted v2 (install executable, or switch to pac
kage minted2) on input line 215.
[...]

[gpoore]

@romildo Yes, I meant latexminted, but it looks like you have everything configured correctly for that. Can you try removing minted2.sty (so there is no fallback), and then looking for a *.minted file in the document directory after compiling? If you can post that file plus the log that goes with it, I may be able to figure out what is causing the issue.

[romildo]

$ pdflatex --shell-escape minted-test
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023/nixos.org) (preloaded format=pdflatex)
 \write18 enabled.
entering extended mode
(./minted-test.tex
LaTeX2e <2023-11-01> patch level 1
L3 programming layer <2024-02-20>

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/article.cls
Document Class: article 2023/05/17 v1.4n Standard LaTeX document class

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/size10.clo))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/inputenc.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/fontenc.sty) (/home/romildo/texmf/tex/latex/minted/minted.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/catchfile/catchfile.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/infwarerr/infwarerr.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/ltxcmds/ltxcmds.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/etexcmds/etexcmds.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/iftex/iftex.sty)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/etoolbox/etoolbox.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/framed/framed.sty) (/home/romildo/texmf/tex/latex/fvextra/fvextra.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/fancyvrb/fancyvrb.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics/keyval.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/upquote/upquote.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/textcomp.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/lineno/lineno.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvoptions/kvoptions.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvsetkeys/kvsetkeys.sty))))
(/home/romildo/texmf/tex/latex/latex2pydata/latex2pydata.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pdftexcmds/pdftexcmds.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgf/utilities/pgfkeys.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeys.code.tex
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeyslibraryfiltered.code.tex)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgfopts/pgfopts.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/tools/shellesc.sty)system returned with code 256


Package minted Warning: minted v3+ Python executable is not installed or is not
 added to PATH; falling back to minted v2 (install executable, or switch to pac
kage minted2) on input line 215.


! LaTeX Error: File `minted2.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name:
! Undefined control sequence.
<recently read> \DeleteFile

l.215 \fi

?
! Emergency stop.
<recently read> \DeleteFile

l.215 \fi

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on minted-test.log.

$ ls -AlF
total 32
-rw-r--r-- 1 romildo romildo   98 mai 26 23:35 _3F29AA7400DFA670869ED2F056F07532.data.minted
-rw-r--r-- 1 romildo romildo  768 mai 26 23:35 _3F29AA7400DFA670869ED2F056F07532.errlog.minted
-rw-r--r-- 1 romildo romildo  158 mai 26 23:35 _3F29AA7400DFA670869ED2F056F07532.message.minted
-rw-r--r-- 1 romildo romildo 8430 mai 26 23:36 minted-test.log
-rw-r--r-- 1 romildo romildo  611 mai 26 17:14 minted-test.tex
-rw-r--r-- 1 romildo romildo   94 mai 26 10:03 test.tex

$ cat _3F29AA7400DFA670869ED2F056F07532.data.minted
{
"command": "config",
"jobname": "minted-test",
"timestamp": "20240526233553",
"cachedir": "",
}

$ cat _3F29AA7400DFA670869ED2F056F07532.errlog.minted
File "/nix/store/n3kmpcq4zwpjsmy7immsqz0d2sv8sg31-latexminted-0.1.0b1/lib/python3.11/site-packages/latexminted/cmdline.py", line 95, in main
  data = load_data(md5=md5, messages=messages, timestamp=timestamp, command=cmdline_args.subparser_name)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/n3kmpcq4zwpjsmy7immsqz0d2sv8sg31-latexminted-0.1.0b1/lib/python3.11/site-packages/latexminted/load_data.py", line 26, in load_data
  data_text = data_path.read_text()
              ^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/n3kmpcq4zwpjsmy7immsqz0d2sv8sg31-latexminted-0.1.0b1/lib/python3.11/site-packages/latexminted/restricted/_restricted_pathlib.py", line 203, in read_text
  raise PathSecurityError(

$ cat _3F29AA7400DFA670869ED2F056F07532.message.minted
\def\minted@exec@error{%
  \minted@error{Failed due to unexpected error (see \detokenize{"_3F29AA7400DFA670869ED2F056F07532.errlog.minted"} if it exists)}%
}%

minted-test.log

[gpoore]

@romildo There's a conflict with the security features for restricted shell escape. This might be a conflict with the restriction that files be under the home directory, as resolved by Python's pathlib.Path.home(). You can run

python -c "import pathlib; print(pathlib.Path.home().resolve())"

to see what Python considers your home directory, and then see if your .tex file is under that. This restriction is technically a bit more restrictive than required by TeX Live, so it should be possible to relax somewhat if necessary. It's also possible that I can keep the restriction but need to try more than pathlib.Path.home() to get the correct directory.

[romildo]

The .tex file is not under my home directory. That would be too much restrictive.

$ python -c "import pathlib; print(pathlib.Path.home().resolve())"
/home/romildo

$ pwd
/alt/programming/latex/minted

[romildo]

If I move the .tex file to a subdirectory of my home directory, it works.

$ pwd
/home/romildo/tmp/minted

$ pdflatex --shell-escape minted-test
[...]

$ pdflatex --shell-escape minted-test
[...]

minted-test.log

[romildo]

But if I add the -output-directory option to pdflatex, it does not work anymore:

$ pwd
/home/romildo/tmp/minted

$ mkdir _build

$ pdflatex -output-directory=_build --shell-escape minted-test
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023/nixos.org) (preloaded format=pdflatex)
 \write18 enabled.
entering extended mode
(./minted-test.tex
LaTeX2e <2023-11-01> patch level 1
L3 programming layer <2024-02-20>

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/article.cls
Document Class: article 2023/05/17 v1.4n Standard LaTeX document class

(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/size10.clo))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/inputenc.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/fontenc.sty) (/home/romildo/texmf/tex/latex/minted/minted.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/catchfile/catchfile.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/infwarerr/infwarerr.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/ltxcmds/ltxcmds.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/etexcmds/etexcmds.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/iftex/iftex.sty)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/etoolbox/etoolbox.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/framed/framed.sty) (/home/romildo/texmf/tex/latex/fvextra/fvextra.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/fancyvrb/fancyvrb.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/graphics/keyval.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/upquote/upquote.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/base/textcomp.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/lineno/lineno.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvoptions/kvoptions.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/kvsetkeys/kvsetkeys.sty))))
(/home/romildo/texmf/tex/latex/latex2pydata/latex2pydata.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pdftexcmds/pdftexcmds.sty))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgf/utilities/pgfkeys.sty
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeys.code.tex
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/generic/pgf/utilities/pgfkeyslibraryfiltered.code.tex)))
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/pgfopts/pgfopts.sty)
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/tools/shellesc.sty)Traceback (most recent call last):
  File "/nix/store/n3kmpcq4zwpjsmy7immsqz0d2sv8sg31-latexminted-0.1.0b1/bin/.latexminted-wrapped", line 9, in <module>
    sys.exit(main())
             ^^^^^^
  File "/nix/store/n3kmpcq4zwpjsmy7immsqz0d2sv8sg31-latexminted-0.1.0b1/lib/python3.11/site-packages/latexminted/cmdline.py", line 105, in main
    config(**func_args)
TypeError: config() missing 1 required keyword-only argument: 'data'
system returned with code 256


Package minted Warning: minted v3+ Python executable is not installed or is not
 added to PATH; falling back to minted v2 (install executable, or switch to pac
kage minted2) on input line 215.


! LaTeX Error: File `minted2.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name:
! Undefined control sequence.
<recently read> \DeleteFile

l.215 \fi

?

! LaTeX Error: Missing \begin{document}.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...

l.215 \fi

?
! Emergency stop.
 ...

l.215 \fi

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on _build/minted-test.log.

$ ls -AlF
total 8
drwxr-xr-x 2 romildo romildo 4096 mai 27 09:28 _build/
-rw-r--r-- 1 romildo romildo  611 mai 27 09:19 minted-test.tex

$ ls -AlF _build
total 16
-rw-r--r-- 1 romildo romildo   98 mai 27 09:28 _3F29AA7400DFA670869ED2F056F07532.data.minted
-rw-r--r-- 1 romildo romildo 8816 mai 27 09:28 minted-test.log

$ cat _build/_3F29AA7400DFA670869ED2F056F07532.data.minted
{
"command": "config",
"jobname": "minted-test",
"timestamp": "20240527092805",
"cachedir": "",
}

minted-test.log

[gpoore]

@romildo I've just released latexminted v0.1.0b2. This removes the home directory restriction (which was broader than required) and fixes a bug with -output-directory.

[romildo]

@romildo I've just released latexminted v0.1.0b2. This removes the home directory restriction (which was broader than required) and fixes a bug with -output-directory.

Thanks. Now it works outside of the home directory.

As I do not have texlive 2024 yet, I found that I have to explicitly set the environment variable TEXMF_OUTPUT_DIRECTORY in order to use -output-directory.

[romildo]

@gpoore, with minted v3, using the minted library for tcolorbox gives an error.

\tcbuselibrary{minted}

[...]
(/nix/store/x9crynqyh5ygrfzqbj4cd39dwabx4kk6-texlive-combined-2023-texmfdist/te
x/latex/tcolorbox/tcbminted.code.tex
Library (tcolorbox): 'tcbminted.code.tex' version '6.2.0'
! Undefined control sequence.
l.26 \minted@def@optcl
                    {envname}{-P envname}{#1}
?

[gpoore]

@romildo tcolorbox adds an option to minted. However, with minted v3, option handling is redesigned so this doesn't work. I'll need to open an issue at the tcolorbox repository to get this removed, and make any needed modifications in minted v3 to provide additional functionality. This relates to #281.

[romildo]

@gpoore it seems that escapeinside is not working in minted v3, right?

[gpoore]

@romildo Do you have a specific example of escapeinside not working? As far as I can tell, it technically works, but is much more limited than the last time I used it. The examples in the documentation no longer work, but it does work in comments. My guess is that something has changed in Pygments, either in some of the lexers or perhaps in the LaTeX formatter, that prevents escapeinside from working in cases where it did previously.

[romildo]

@gpoore Here is an example of escapeinside not working:

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{minted}

\begin{document}

\begin{minted}[frame=single, escapeinside=!!]{c}
// defining !\underline{pi}!:
const double !\underline{pi}! = 3.1415926535;
\end{minted}

\end{document}

$ grep 'Package: minted' test.log
Package: minted 2024/06/11 v3.0.0beta5 Yet another Pygments shim for LaTeX

[romildo]

With minted2, it works:

$ grep 'Package: minted' test.log
Package: minted2 2024/05/09 v2.9 Yet another Pygments shim for LaTeX

[gpoore]

@romildo It should work now with the latest commit. The Pygments lexer needed to be wrapped in LatexEmbeddedLexer for escapes to work everywhere; passing escapeinside to LatexFormatter only enabled escapes in comments.

[gpoore]

@romildo The latest minted v3 beta with the latest dev version of fvextra adds tcolorbox compatibility.

[muzimuzhi]

I'll need to open an issue at the tcolorbox repository to get this removed, and make any needed modifications in minted v3 to provide additional functionality. This relates to #281.

The latest minted v3 beta with the latest dev version of fvextra adds tcolorbox compatibility.

Thanks. I've informed the maintainer of tcolorbox of this change, see T-F-S/tcolorbox#2 (comment).

[romildo]

@gpoore I am trying to run the latest version of latexminted on my NixOS system, but if fails:

$ latexminted --version
Traceback (most recent call last):
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/_latex_config.py", line 440, in TEXMFOUTPUT
    value = self._var_cache['TEXMFOUTPUT']
            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'TEXMFOUTPUT'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/bin/.latexminted-wrapped", line 6, in <module>
    from latexminted.cmdline import main
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/cmdline.py", line 13, in <module>
    from .restricted import os_environ
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/__init__.py", line 12, in <module>
    from ._restricted_pathlib import RestrictedPath
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/_restricted_pathlib.py", line 25, in <module>
    if latex_config.TEXMFOUTPUT:
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/_latex_config.py", line 443, in TEXMFOUTPUT
    if value is None and self.texlive_kpsewhich:
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/_latex_config.py", line 199, in texlive_kpsewhich
    self._init_tex_paths()
  File "/nix/store/fl4qqwhqgd0jhiwjkl2fhgjba49127f3-latexminted-0.1.0b9/lib/python3.12/site-packages/latexminted/restricted/_latex_config.py", line 181, in _init_tex_paths
    raise LatexConfigError(
latexminted.err.LatexConfigError: Expected environment variables SELFAUTOLOC and/or TEXSYSTEM were not found or had invalid values, so "kpsewhich" executable and/or MiKTeX "initexmf" executable could not be found

$ kpsewhich --version
kpathsea version 6.3.5
Copyright 2023 Karl Berry & Olaf Weber.
License LGPLv2.1+: GNU Lesser GPL version 2.1 or later <https://gnu.org/licenses/lgpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Any clues?

[gpoore]

@romildo I think you're a few versions behind. The current version is 0.1.0b15. This requires the new latexrestricted library.

[romildo]

@romildo I think you're a few versions behind. The current version is 0.1.0b15. This requires the new latexrestricted library.

(Sorry for that. I had just booted into the wrong system.)

It is still not working:

$ latexminted --version
Traceback (most recent call last):
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_restricted_pathlib.py", line 647, in writable_file
    return self._writable_file_cache[self.cache_key]
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: (<class 'latexrestricted._restricted_pathlib.ResolvedRestrictedPath'>, ResolvedRestrictedPath('/home/romildo/.latexminted_config'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/bin/.latexminted-wrapped", line 6, in <module>
    from latexminted.cmdline import main
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/cmdline.py", line 13, in <module>
    from .restricted import os_environ
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/restricted/__init__.py", line 13, in <module>
    from ._latexminted_config import latexminted_config
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/restricted/_latexminted_config.py", line 218, in <module>
    latexminted_config = LatexMintedConfig()
                         ^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/restricted/_latexminted_config.py", line 116, in __init__
    self._load(LatexMintedConfigPath.home() / config_name)
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/restricted/_latexminted_config.py", line 140, in _load
    data_str = path.read_text(encoding='utf-8-sig')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_restricted_pathlib.py", line 202, in read_text
    is_readable, reason = self.readable_file()
                          ^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fyvbk8dn2pwyk2rq0idvw3wvds7b25zg-latexminted-0.1.0b15/lib/python3.12/site-packages/latexminted/restricted/_latexminted_config.py", line 42, in readable_file
    is_latex_writable, _ = ResolvedRestrictedPath(self).writable_file()
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_restricted_pathlib.py", line 661, in writable_file
    if self.can_write_dotfiles() or not any(p.name.startswith('.') for p in (self, resolved)):
       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_restricted_pathlib.py", line 124, in can_write_dotfiles
    return latex_config.can_write_dotfiles
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_latex_config.py", line 469, in can_write_dotfiles
    self._init_write_settings()
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_latex_config.py", line 438, in _init_write_settings
    cls._init_tex_paths()
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_latex_config.py", line 163, in _init_tex_paths
    cls._init_tex_paths_texlive()
  File "/nix/store/vrdb3rg0c3zdzzc06q7glp55qvp5swsz-python3.12-latexrestricted-0.4.0/lib/python3.12/site-packages/latexrestricted/_latex_config.py", line 205, in _init_tex_paths_texlive
    raise LatexConfigError('Environment variable SELFAUTOLOC is expected for TeX Live, but was not set')
latexrestricted.err.LatexConfigError: Environment variable SELFAUTOLOC is expected for TeX Live, but was not set

$ kpsewhich -var-value=SELFAUTOLOC
/nix/store/d4lq8nwx70i8is18rgcw8mvqwjd6aq9z-texlive-combined-2023/bin

[gpoore]

@romildo I just uploaded v0.1.0b16 to GitHub and PyPI. That should fix things. I've been running latexminted within a TeX Live installation, which was setting a SELFAUTOLOC environment variable and preventing me from seeing this bug. You will need the dev version of fvextra.sty for full functionality.

[romildo]

@gpoore It seems that restricted shell escape is not working anymore on my system. Adding the option -shell-escape in the command line, it works.

$ pwd
/home/romildo/tmp/minted

$ python -c "import pathlib; print(pathlib.Path.home().resolve())"
/home/romildo

$ pdflatex test.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023/nixos.org) (preloaded format=pdflatex)
 restricted \write18 enabled.
[...]
! Package minted Error: minted v3+ executable is not installed, is not added to
 PATH, or is not permitted with restricted shell escape.
[...]

$ pdflatex -shell-escape test.tex
[...]
Output written on test.pdf (1 page, 12530 bytes).
Transcript written on test.log.

$ grep 'Package: minted' test.log
Package: minted 2024/08/27 v3.0.0beta13 Yet another Pygments shim for LaTeX

[gpoore]

@romildo Have you modified your texmf.cnf to add latexminted to shell_escape_commands? I'm waiting on a security review for TeX Live to complete so that TeX Live will automatically include latexminted in the list of trusted programs, but until then you have to add it to the list of trusted programs manually.