[AUCTeX] TeX-auto-generate confused by comments

[Tristan Miller]

Dear all,

In a separate thread I was advised to run TeX-auto-generate to generate
style hooks for the custom classes I'm using. I discovered that this
command seems to get confused by comments in the optional argument to
\RequirePackage. For example, take the following class file:

\NeedsTeXFormat{LaTeX2e}
\LoadClassWithOptions{article}
\ProvidesClass{foo}[2017/09/07 v0.1 test class]
\RequirePackage[
backend=biber, % here is a comment
]{biblatex}

Running TeX-auto-generate on this class file results in the following
style file, which fails to account for the use of biblatex:

(TeX-add-style-hook
"foo"
(lambda ()
(TeX-run-style-hooks
"latex2e"
"article"
"art10"))
:latex)

However, removing "% here is a comment" and re-running
TeX-auto-generate results in the following, which looks more correct:

(TeX-add-style-hook
"foo"
(lambda ()
(TeX-add-to-alist 'LaTeX-provided-package-options
'(("biblatex" "backend=biber" "")))
(TeX-run-style-hooks
"latex2e"
"article"
"art10"
"biblatex"))
:latex)

Am I correct in assuming that this is a bug? I know that a lot of
people use comments like this to explain the purpose of (sometimes very
obscure) package options.

Regards,
Tristan

[Ikumi Keita]

Dear Tristan,

Post by Tristan Miller
Dear all,
In a separate thread I was advised to run TeX-auto-generate to generate
style hooks for the custom classes I'm using. I discovered that this
command seems to get confused by comments in the optional argument to
\NeedsTeXFormat{LaTeX2e}
\LoadClassWithOptions{article}
\ProvidesClass{foo}[2017/09/07 v0.1 test class]
\RequirePackage[
backend=biber, % here is a comment
]{biblatex}
Running TeX-auto-generate on this class file results in the following
(TeX-add-style-hook
"foo"
(lambda ()
(TeX-run-style-hooks
"latex2e"
"article"
"art10"))
:latex)
However, removing "% here is a comment" and re-running
(TeX-add-style-hook
"foo"
(lambda ()
(TeX-add-to-alist 'LaTeX-provided-package-options
'(("biblatex" "backend=biber" "")))
(TeX-run-style-hooks
"latex2e"
"article"
"art10"
"biblatex"))
:latex)
Am I correct in assuming that this is a bug? I know that a lot of
people use comments like this to explain the purpose of (sometimes very
obscure) package options.

IMHO, a style file generated by `TeX-auto-generate' should be considered
as just a "start point" to begin your customize with. The function does
nothing more than mere regexp matchings and is not a real TeX parser
after all. Generally speaking, you have to edit the generated file as
you described and move it from "auto" directory to "style"
directory. (e.g. from "~/.emacs.d/auctex/auto/" to
"~/.emacs.d/auctex/style/")

Having said that, it seems to my eyes that `LaTeX-auto-cleanup' in
latex.el is written to be capable of parsing optional package options
which contain comments like that... Well, tuning the regular expression
for "RequirePackage" would do the trick. Find the part
---------------------------------------------------------------
(defvar LaTeX-auto-class-regexp-list
'(;; \RequirePackage[<options>]{<package>}[<date>]
("\\\\Require\\(Package\\)\\(\\[\\([^#\\.%]*?\\)\\]\\)?\
{\\([^#\\.\n\r]+?\\)}"
...
---------------------------------------------------------------
in latex.el of AUCTeX and replace "[^#\\.%]*?" with "[^\]\\]*", the same
as the corresponding segment of regexp for usepackage. Then save it and
byte-compile. After that, restart emacs and try `TeX-auto-generate'
on "foo.cls" again. I expect that the generated style file is
----------------------------------------------------------------------
(TeX-add-style-hook
"foo"
(lambda ()
(TeX-add-to-alist 'LaTeX-provided-package-options
'(("biblatex" "backend=biber" "")))
(add-to-list 'LaTeX-verbatim-macros-with-braces-local "url")
(add-to-list 'LaTeX-verbatim-macros-with-braces-local "path")
(add-to-list 'LaTeX-verbatim-macros-with-delims-local "url")
(add-to-list 'LaTeX-verbatim-macros-with-delims-local "path")
(TeX-run-style-hooks
"latex2e"
"article"
"art10"
"biblatex"))
:latex)
----------------------------------------------------------------------
and serves your purpose. The lines concerned with "url" and "path" are
redundant and dispensable in this particular case, but must be harmless.

Regards,
Ikumi Keita

[Tristan Miller]

Greetings.

On Sat, 09 Sep 2017 02:00:44 +0900, Ikumi Keita

Post by Ikumi Keita
IMHO, a style file generated by `TeX-auto-generate' should be
considered as just a "start point" to begin your customize with. The
function does nothing more than mere regexp matchings and is not a
real TeX parser after all.

Sure, but commented package options are a common and syntactically
simple enough case that I would have expected TeX-auto-generate to
handle them properly.

Post by Ikumi Keita
Well, tuning the regular
expression for "RequirePackage" would do the trick. Find the part
---------------------------------------------------------------
(defvar LaTeX-auto-class-regexp-list
'(;; \RequirePackage[<options>]{<package>}[<date>]
("\\\\Require\\(Package\\)\\(\\[\\([^#\\.%]*?\\)\\]\\)?\
{\\([^#\\.\n\r]+?\\)}"
...
---------------------------------------------------------------
in latex.el of AUCTeX and replace "[^#\\.%]*?" with "[^\]\\]*", the
same as the corresponding segment of regexp for usepackage.

Yes, that fixes the problem. Thank you!

Is there a reason why \RequirePackage uses a different regular
expression than \usepackage? If not, then perhaps the AUCTeX
maintainers might consider adopting the change you've given here.

Regards,
Tristan

[Ikumi Keita]

Hi Tristan,

Post by Tristan Miller
Is there a reason why \RequirePackage uses a different regular
expression than \usepackage?

I have no idea. According to the commit log, they have been different
since the first introduction of `LaTeX-auto-class-regexp-list' in 2004.
Does anyone know?

Post by Tristan Miller
If not, then perhaps the AUCTeX maintainers might consider adopting
the change you've given here.

I would commit to the repository if it is confirmed to be safe to change
the regular expression as discussed here.

Regards,
Ikumi Keita

[Tassilo Horn]

Ikumi Keita <***@ikumi.que.jp> writes:

Hi Ikumi-san,

Post by Ikumi Keita

Post by Tristan Miller
If not, then perhaps the AUCTeX maintainers might consider adopting
the change you've given here.

I would commit to the repository if it is confirmed to be safe to
change the regular expression as discussed here.

I don't see a reason why it shouldn't be safe, so please go ahead.

Bye,
Tassilo

[Mosè Giordano]

Hi Tassilo and Keita,

Post by Tassilo Horn
Hi Ikumi-san,

Post by Ikumi Keita

Post by Tristan Miller
If not, then perhaps the AUCTeX maintainers might consider adopting
the change you've given here.

I would commit to the repository if it is confirmed to be safe to
change the regular expression as discussed here.

I don't see a reason why it shouldn't be safe, so please go ahead.

Green light also from me, if the patch comes with a test :-)

Bye,
Mosè

[Ikumi Keita]

Hi all,

Post by Mosè Giordano

Post by Tassilo Horn
I don't see a reason why it shouldn't be safe, so please go ahead.

Green light also from me, if the patch comes with a test :-)

I was cautious because the current regexp "[^#\.%]*?" excludes "#" and
"." as well as "%" from the option string while "[^]\]*" does not.
Now I see I don't have to care about that, I'm going to commit the
change with a test. I will apply the same change on the regexp for
LoadClass.

Thanks,
Ikumi Keita

[Arash Esbati]

Post by Mosè Giordano

Post by Tassilo Horn
Hi Ikumi-san,

Post by Ikumi Keita

Post by Tristan Miller
If not, then perhaps the AUCTeX maintainers might consider adopting
the change you've given here.

I would commit to the repository if it is confirmed to be safe to
change the regular expression as discussed here.

I don't see a reason why it shouldn't be safe, so please go ahead.

Green light also from me, if the patch comes with a test :-)

Hi all,

this change was overdue, thanks Keita. I was looking at the definition
of `TeX-add-to-alist'[1] and it uses `add-to-list' to add elements to
local-bound variables; I learned here that this should be avoided. Any
takers?

Best, Arash

Footnotes:
[1] http://git.savannah.gnu.org/cgit/auctex.git/tree/tex.el#n4822