[Enhancement] Search for filters/lua-filters within sub-directories of $DATADIR/filters

[the-solipsist]

Problem:
Currently, if a lua-filter (say wordcount.lua) is at $DATADIR/filters/wordcount.lua, things work fine. But if it is in $DATADIR/filters/wordcount/wordcount.lua, then the pandoc command is unable to locate it.

Commands:

$ cd $HOME/.local/share
$ git clone https://github.com/pandoc/lua-filters.git pandoc/filters
$ pandoc --lua-filter=wordcount.lua ~/example.md

Expected:

311 words in body
1741 characters in body
2014 characters in body (including spaces)

Actual Output:

Error running filter wordcount.lua:
wordcount.lua: openBinaryFile: does not exist (No such file or directory)

Workaround:

$ pandoc --lua-filter=wordcount/wordcount.lua ~/example.md

The relevant code seems to be in this file:

pandoc/src/Text/Pandoc/Filter/Path.hs

Lines 22 to 35 in 93e3d46

	expandFilterPath :: PandocMonad m => FilePath -> m FilePath
	expandFilterPath fp = do
	mbDatadir <- getUserDataDir
	fpExists <- fileExists fp
	if fpExists
	then return fp
	else case mbDatadir of
	Just datadir | isRelative fp -> do
	let filterPath = datadir </> "filters" </> fp
	filterPathExists <- fileExists filterPath
	if filterPathExists
	then return filterPath
	else return fp
	_ -> return fp

Please close this issue if it isn't worth changing, since the "workaround" works well enough. However, some software (for instance prosody wrt to it modules) would allow you to refer to "wordcount/wordcount.lua" as "wordcount" (dropping both the directory name as well as .lua, as that's the extension it would be looking for).

[argent0]

Hi, what would happen If you have two files with the same name under different sub-folders ?

[the-solipsist]

Hi, @argent0. Prosody uses very simple logic: if you ask it to load {modulename} it either looks for {modulename}/{modulename}.lua or {modulename}.lua within the module paths that you've set[1]. So, something similar can be done here too, given that the the lua-filters git directory is also organized as {filtername}/{filtername}.lua.

So, to answer your question, as per my proposed logic, Pandoc would only recognize a file if it is either in the base directory, or in a sub-directory with the same name as the file.

[1]: For simplicity's sake, I'm discounting the optional "mod_" that Prosody uses for identifying modules. For the exact code, see the link I've provided to "pluginloader.lua".

[tarleb]

I'm sympathetic with the request, it would make using the lua-filters repo much easier. However, I'm wary of the complexity that comes with such a feature, so I lean towards treating it as out of scope for pandoc.

[tarleb]

I've changed my stance on this and opened #8822. My reasoning is that I'd rather see people use symbolic links to the filter file; however, Windows doesn't work that well with symbolic links, and even on posix it's often convenient to simply place a fulle git subtrees in the data dir.