Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-33616

TeX filter should work with a parameter --halt-on-error

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.2
    • Fix Version/s: 2.4
    • Component/s: Maths filters
    • Labels:
    • Testing Instructions:
      Hide

      Prerequisite

      1. Install latex, dvips and convert libraries
        On ubuntu use
        sudo apt-get install texlive-full (This will install all packages in LaTeX distribution)
      1. Login as admin
      2. Enable tex filter (Settings -> Site administration -> Plugins -> Filters -> Manage filters)
      3. Click settings (next to tex filter) and make sure libraries point to right path
      4. Create a label in course paste "$$ \sqrt {x + y}

        $$" or refer http://docs.moodle.org/22/en/Using_TeX_Notation_2 for other tex symbols

      5. Save and return to course
      6. Make sure you can see square-root x + y image.
      Show
      Prerequisite Install latex, dvips and convert libraries On ubuntu use sudo apt-get install texlive-full (This will install all packages in LaTeX distribution) Login as admin Enable tex filter (Settings -> Site administration -> Plugins -> Filters -> Manage filters) Click settings (next to tex filter) and make sure libraries point to right path Create a label in course paste "$$ \sqrt {x + y} $$" or refer http://docs.moodle.org/22/en/Using_TeX_Notation_2 for other tex symbols Save and return to course Make sure you can see square-root x + y image.
    • Workaround:
      Hide

      Add the --halt-on-error parameter after --interaction=nonstopmode on line 111 of /filter/tex/latex.php and on line 247 of /filter/tex/texdebug.php.

      Show
      Add the --halt-on-error parameter after --interaction=nonstopmode on line 111 of /filter/tex/latex.php and on line 247 of /filter/tex/texdebug.php.
    • Affected Branches:
      MOODLE_22_STABLE
    • Fixed Branches:
      MOODLE_24_STABLE
    • Pull Master Branch:
      wip-mdl-33616

      Description

      My provider tells me that some mistakes in the TeX code in one course result massive server activity. The TeX plugin calls the LaTeX functions with a parameter --interaction=nonstopmode so an error would not stop the execution of the function and the user does not get a message about his mistake. I don't know if this writing is totally correct. My provider wants to disallow the TeX filter because the TeX filter seems to be a security leak.

      The provider says that the TeX filter should work with a parameter --halt-on-error when it calls the LaTeX system functions.

      Please tell me what I should do!

      Ralf

        Gliffy Diagrams

        1. tex.patch
          1 kB
          Michael de Raadt
        1. latex-setting.png
          90 kB

          Issue Links

            Activity

            Hide
            salvetore Michael de Raadt added a comment -

            I'll predicate this by saying I'm no expert on TeX. However if a process is running indefinitely unchecked, that's not great.

            I had a quick search and found that we should perhaps be using these parameters together...

            --interaction=nonstopmode --halt-on-error

            I added --halt-on-error to the filter TeX commands and, well, it didn't break. I tested this using the DragMath editor in a page with the TeX notation filter turned on.

            I'm not sure how I would test this to ensure it stops when an error is encountered. Perhaps someone can suggest an erroneous piece of LaTeX we could throw at it.

            Show
            salvetore Michael de Raadt added a comment - I'll predicate this by saying I'm no expert on TeX. However if a process is running indefinitely unchecked, that's not great. I had a quick search and found that we should perhaps be using these parameters together... --interaction=nonstopmode --halt-on-error I added --halt-on-error to the filter TeX commands and, well, it didn't break. I tested this using the DragMath editor in a page with the TeX notation filter turned on. I'm not sure how I would test this to ensure it stops when an error is encountered. Perhaps someone can suggest an erroneous piece of LaTeX we could throw at it.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Adding this to current sprint.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Adding this to current sprint.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            I am not sure if --halt-on-error should be added to core, as this will halt tex operation on the first error it encounters. This might break any page which has wrong tex content.

            Workaround: Suffix --halt-on-error in Path of latex binary (Settings -> Site administration -> Plugins -> Filters -> Manage filters)

            /usr/bin/latex --halt-on-error

            Ralf, please let us know if this solves your issue.

            Show
            rajeshtaneja Rajesh Taneja added a comment - I am not sure if --halt-on-error should be added to core, as this will halt tex operation on the first error it encounters. This might break any page which has wrong tex content. Workaround: Suffix --halt-on-error in Path of latex binary (Settings -> Site administration -> Plugins -> Filters -> Manage filters) /usr/bin/latex --halt-on-error Ralf, please let us know if this solves your issue.
            Hide
            krause Ralf Krause added a comment -

            Hi,

            when I write the parameter into the configuration field I get an error with the LaTeX path. Please look at the screenshot.

            Ralf

            Show
            krause Ralf Krause added a comment - Hi, when I write the parameter into the configuration field I get an error with the LaTeX path. Please look at the screenshot. Ralf
            Hide
            krause Ralf Krause added a comment -

            As I understand my support team they need to stop any wrong tex statement. They found that one wrong statement results a flood of unterminated function calls and big stress for the whole server.

            I don't know if it would be correct that every tex formula starts its own latex process. If a tex formula is not working the browser does not get the image for the page and the browser tries to get it once more ... I don't know how often my browser tries to get the images? I think the browser is waiting for a timeout and then it asks for the image again and again.

            The supporters intend that the broken tex formula doesn't start a lot of unterminated processes. All other tex formulas should work on the page if they are written in a correct syntax.

            Ralf

            Show
            krause Ralf Krause added a comment - As I understand my support team they need to stop any wrong tex statement. They found that one wrong statement results a flood of unterminated function calls and big stress for the whole server. I don't know if it would be correct that every tex formula starts its own latex process. If a tex formula is not working the browser does not get the image for the page and the browser tries to get it once more ... I don't know how often my browser tries to get the images? I think the browser is waiting for a timeout and then it asks for the image again and again. The supporters intend that the broken tex formula doesn't start a lot of unterminated processes. All other tex formulas should work on the page if they are written in a correct syntax. Ralf
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks for quick response Ralf.

            I know LaTeX path will show error, but it's false true. --halt-on-error being a switch/option for LaTeX should work.
            If you suffix --halt-on-error to path, the command at line 111 - latex.php will become

            /usr/bin/latex --halt-on-error --interaction=nonstopmode /var/moodledata/m/temp/latex/e7b05b32ebc3cdc8e5f26fb8ddab2c88.tex

            AFAIK sequence of option/switch doesn't matter, so it should work in your case.

            On Line 114 - latex.php, return has been commented. If you are not on windows, I suggest you to uncomment this line and try. As this should solve the issue. Also, it will be helpful, if can you please share broken tex formula for testing.

            It will be helpful if you can please try and let us know if either workaround works for you.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks for quick response Ralf. I know LaTeX path will show error, but it's false true. --halt-on-error being a switch/option for LaTeX should work. If you suffix --halt-on-error to path, the command at line 111 - latex.php will become /usr/bin/latex --halt-on-error --interaction=nonstopmode /var/moodledata/m/temp/latex/e7b05b32ebc3cdc8e5f26fb8ddab2c88.tex AFAIK sequence of option/switch doesn't matter, so it should work in your case. On Line 114 - latex.php, return has been commented. If you are not on windows, I suggest you to uncomment this line and try. As this should solve the issue. Also, it will be helpful, if can you please share broken tex formula for testing. It will be helpful if you can please try and let us know if either workaround works for you.
            Hide
            poltawski Dan Poltawski added a comment -

            We need some real data here in order to make a decision:

            • What tex commands are causing problems.
            • What tex commands are OK? Are they affected?

            Note that I think the combination of "-halt-on-error --interaction=nonstopmode" looks safe and sensible to me. If you can find a combination of tex statements which would be halted and previously would've generated an image then maybe we can make a better informed descision.

            If it does cause some tex images to be generated then i'm pretty sure it would be limited to an individual tex image.

            Show
            poltawski Dan Poltawski added a comment - We need some real data here in order to make a decision: What tex commands are causing problems. What tex commands are OK? Are they affected? Note that I think the combination of "-halt-on-error --interaction=nonstopmode" looks safe and sensible to me. If you can find a combination of tex statements which would be halted and previously would've generated an image then maybe we can make a better informed descision. If it does cause some tex images to be generated then i'm pretty sure it would be limited to an individual tex image.
            Hide
            krause Ralf Krause added a comment - - edited

            Hi Rajesh, hi Dan,

            I have three test servers with the same Tex course in Moodle

            http://fobi.schulon.org/course/view.php?id=367 - Moodle 2.2.1+ (Build: 20120213)
            http://www3.edumoodle.at/duesseldorf/course/view.php?id=7 - Moodle 2.1.3+ (Build: 20111209)
            http://www.moodletreff.de/course/view.php?id=294 - Moodle 2.2.3+ (Build: 20120615)

            I all three cases I don't know anything about their server software and their LaTeX installation.

            The used LaTeX preamble is always the same

            \usepackage[latin1]{inputenc}
            \usepackage{amsmath}
            \usepackage{amsfonts}
            \usepackage{color}
            \RequirePackage{amsmath,amssymb,latexsym}

            The Moodletreff server uses a self compiled mimeTeX because they had problems with the LaTeX installation guide shown in the Moodle docs.

            The EduMoodle.at and the schulon.org server use LaTeX ... the shown path in the TeX strings is as following

            /usr/bin/latex

            Until last week the EduMoodle.at server showed the results for \tiny, \small, \large, \Large, and \LARGE in a correct size ... since this week there are no differences ... bad result. Also the colored formulas in topic 12 were correct last week.

            The schulon.org server does not show gif images because in this case I tested the additional parameter --halt-on-error and I get an error (x at the end of the parameter field)

            I hope you will see the differences ...

            Ralf

            Show
            krause Ralf Krause added a comment - - edited Hi Rajesh, hi Dan, I have three test servers with the same Tex course in Moodle http://fobi.schulon.org/course/view.php?id=367 - Moodle 2.2.1+ (Build: 20120213) http://www3.edumoodle.at/duesseldorf/course/view.php?id=7 - Moodle 2.1.3+ (Build: 20111209) http://www.moodletreff.de/course/view.php?id=294 - Moodle 2.2.3+ (Build: 20120615) I all three cases I don't know anything about their server software and their LaTeX installation. The used LaTeX preamble is always the same \usepackage[latin1]{inputenc} \usepackage{amsmath} \usepackage{amsfonts} \usepackage{color} \RequirePackage{amsmath,amssymb,latexsym} The Moodletreff server uses a self compiled mimeTeX because they had problems with the LaTeX installation guide shown in the Moodle docs. The EduMoodle.at and the schulon.org server use LaTeX ... the shown path in the TeX strings is as following /usr/bin/latex Until last week the EduMoodle.at server showed the results for \tiny, \small, \large, \Large, and \LARGE in a correct size ... since this week there are no differences ... bad result. Also the colored formulas in topic 12 were correct last week. The schulon.org server does not show gif images because in this case I tested the additional parameter --halt-on-error and I get an error (x at the end of the parameter field) I hope you will see the differences ... Ralf
            Hide
            krause Ralf Krause added a comment - - edited

            The admin from schulon.org send me the informations about the system software.
            I think that should be okay but the TeX filter is not working correctly.

            fobi.schulon.org: 
             
            Debian/Squeeze 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux 
             
            ii  latex-beamer                        3.07-2                       LaTeX class to produce presentations 
            ii  latex-xcolor                        2.11-1                       Easy driver-independent TeX class for color 
            ii  libdjvulibre-text                   3.5.23-3                     Linguistic support files for libdjvulibre 
            ii  liblocale-gettext-perl              1.05-6                       Using libc functions for internationalization in Perl 
            ii  libtext-charwidth-perl              0.04-6                       get display widths of characters on the terminal 
            ii  libtext-iconv-perl                  1.7-2                        converts between character sets in Perl 
            ii  libtext-wrapi18n-perl               0.06-7                       internationalized substitute of Text::Wrap 
            ii  luatex                              0.60.2-1                     next generation TeX engine 
            ii  tex-common                          2.08.1                       common infrastructure for building and installing TeX 
            ii  texlive                             2009-11+squeeze1             TeX Live: A decent selection of the TeX Live packages 
            ii  texlive-base                        2009-11                      TeX Live: Essential programs and files 
            ii  texlive-binaries                    2009-8                       Binaries for TeX Live 
            ii  texlive-common                      2009-11                      TeX Live: Base component 
            ii  texlive-doc-base                    2009-2                       TeX Live: TeX Live documentation 
            ii  texlive-extra-utils                 2009-10                      TeX Live: TeX auxiliary programs 
            ii  texlive-font-utils                  2009-10                      TeX Live: TeX and Outline font utilities 
            ii  texlive-fonts-recommended           2009-11+squeeze1             TeX Live: Recommended fonts 
            ii  texlive-fonts-recommended-doc       2009-11+squeeze1             TeX Live: Documentation files for texlive-fonts-recommended 
            ii  texlive-generic-recommended         2009-11+squeeze1             TeX Live: Recommended generic packages 
            ii  texlive-latex-base                  2009-11                      TeX Live: Basic LaTeX packages 
            ii  texlive-latex-base-doc              2009-11                      TeX Live: Documentation files for texlive-latex-base 
            ii  texlive-latex-recommended           2009-11+squeeze1             TeX Live: LaTeX recommended packages 
            ii  texlive-latex-recommended-doc       2009-11+squeeze1             TeX Live: Documentation files for texlive-latex-recommended 
            ii  texlive-luatex                      2009-11                      TeX Live: LuaTeX packages 
            ii  texlive-metapost                    2009-11+squeeze1             TeX Live: MetaPost (and Metafont) drawing packages 
            ii  texlive-metapost-doc                2009-11+squeeze1             TeX Live: Documentation files for texlive-metapost 
            ii  texlive-pstricks                    2009-10                      TeX Live: PSTricks packages 
            ii  texlive-pstricks-doc                2009-10                      TeX Live: Documentation files for texlive-pstricks 

            Show
            krause Ralf Krause added a comment - - edited The admin from schulon.org send me the informations about the system software. I think that should be okay but the TeX filter is not working correctly. fobi.schulon.org:   Debian/Squeeze 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux   ii latex-beamer 3.07-2 LaTeX class to produce presentations ii latex-xcolor 2.11-1 Easy driver-independent TeX class for color ii libdjvulibre-text 3.5.23-3 Linguistic support files for libdjvulibre ii liblocale-gettext-perl 1.05-6 Using libc functions for internationalization in Perl ii libtext-charwidth-perl 0.04-6 get display widths of characters on the terminal ii libtext-iconv-perl 1.7-2 converts between character sets in Perl ii libtext-wrapi18n-perl 0.06-7 internationalized substitute of Text::Wrap ii luatex 0.60.2-1 next generation TeX engine ii tex-common 2.08.1 common infrastructure for building and installing TeX ii texlive 2009-11+squeeze1 TeX Live: A decent selection of the TeX Live packages ii texlive-base 2009-11 TeX Live: Essential programs and files ii texlive-binaries 2009-8 Binaries for TeX Live ii texlive-common 2009-11 TeX Live: Base component ii texlive-doc-base 2009-2 TeX Live: TeX Live documentation ii texlive-extra-utils 2009-10 TeX Live: TeX auxiliary programs ii texlive-font-utils 2009-10 TeX Live: TeX and Outline font utilities ii texlive-fonts-recommended 2009-11+squeeze1 TeX Live: Recommended fonts ii texlive-fonts-recommended-doc 2009-11+squeeze1 TeX Live: Documentation files for texlive-fonts-recommended ii texlive-generic-recommended 2009-11+squeeze1 TeX Live: Recommended generic packages ii texlive-latex-base 2009-11 TeX Live: Basic LaTeX packages ii texlive-latex-base-doc 2009-11 TeX Live: Documentation files for texlive-latex-base ii texlive-latex-recommended 2009-11+squeeze1 TeX Live: LaTeX recommended packages ii texlive-latex-recommended-doc 2009-11+squeeze1 TeX Live: Documentation files for texlive-latex-recommended ii texlive-luatex 2009-11 TeX Live: LuaTeX packages ii texlive-metapost 2009-11+squeeze1 TeX Live: MetaPost (and Metafont) drawing packages ii texlive-metapost-doc 2009-11+squeeze1 TeX Live: Documentation files for texlive-metapost ii texlive-pstricks 2009-10 TeX Live: PSTricks packages ii texlive-pstricks-doc 2009-10 TeX Live: Documentation files for texlive-pstricks
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks for the information Ralf, I will read more to see what can be the best solution to solve this issue.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks for the information Ralf, I will read more to see what can be the best solution to solve this issue.
            Hide
            rajeshtaneja Rajesh Taneja added a comment - - edited

            Hello Ralf,

            I am not sure what is happening there, but it seems amsfonts is causing this problem. I replaced LaTex preamble with following and it works fine. As moodle fall-back on mimeTex (if latex is not found) and it works, you can try remove Path to latex binary (make it empty and save) and it should help.

            \usepackage[latin1]{inputenc}
            \usepackage{amsmath}
            \usepackage{accfonts}
            \RequirePackage{amsmath,amssymb,latexsym}

            I am still trying to figure out the right reason here, but it will be helpful if you can test above.

            FYI: Not sure if debian has amsfonts http://packages.debian.org/sid/all/texlive-font-utils/filelist

            You can try other fonts listed here

            Show
            rajeshtaneja Rajesh Taneja added a comment - - edited Hello Ralf, I am not sure what is happening there, but it seems amsfonts is causing this problem. I replaced LaTex preamble with following and it works fine. As moodle fall-back on mimeTex (if latex is not found) and it works, you can try remove Path to latex binary (make it empty and save) and it should help. \usepackage[latin1]{inputenc} \usepackage{amsmath} \usepackage{accfonts} \RequirePackage{amsmath,amssymb,latexsym} I am still trying to figure out the right reason here, but it will be helpful if you can test above. FYI: Not sure if debian has amsfonts http://packages.debian.org/sid/all/texlive-font-utils/filelist You can try other fonts listed here
            Hide
            krause Ralf Krause added a comment - - edited

            Now I got the informations about the edumoodle.at server in Austria.

            www3.edumoodle.at/duesseldorf
             
            Latex Version:
            /usr/bin/latex -> pdfetex
            pdfetex --version
            „pdfeTeX 3.141592-1.21a-2.2 (Web2C 7.5.4)
            kpathsea version 3.5.4
            Copyright (C) 1997-2004 Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX).
            Kpathsea is copyright (C) 1997-2004 Free Software Foundation, Inc.
            There is NO warranty.  Redistribution of this software is
            covered by the terms of both the pdfeTeX copyright and
            the GNU General Public License.
            For more information about these matters, see the files
            named COPYING and the pdfeTeX source.
            Primary author of pdfeTeX: Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX).
            Kpathsea written by Karl Berry and others.“
             
            Server Version:
            OS: CentOS release 5.8 (Final)
            Kernel: 2.6.18-308.8.2.el5
            Last Update 21.Juni

            Since the last update of this server the TeX filter does not work correctly. You can view the course and you will see that there are problems with some symbols (product, sum, integral, greater than, less then, equal). You also will see that the font size can't change (tiny, small, large, Large, LARGE)

            The admin wrote that the updated the kernel but they did not change anything with the LaTeX installation.

            Show
            krause Ralf Krause added a comment - - edited Now I got the informations about the edumoodle.at server in Austria. www3.edumoodle.at/duesseldorf Latex Version: /usr/bin/latex -> pdfetex pdfetex --version „pdfeTeX 3.141592-1.21a-2.2 (Web2C 7.5.4) kpathsea version 3.5.4 Copyright (C) 1997-2004 Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX). Kpathsea is copyright (C) 1997-2004 Free Software Foundation, Inc. There is NO warranty. Redistribution of this software is covered by the terms of both the pdfeTeX copyright and the GNU General Public License. For more information about these matters, see the files named COPYING and the pdfeTeX source. Primary author of pdfeTeX: Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX). Kpathsea written by Karl Berry and others.“ Server Version: OS: CentOS release 5.8 (Final) Kernel: 2.6.18-308.8.2.el5 Last Update 21.Juni Since the last update of this server the TeX filter does not work correctly. You can view the course and you will see that there are problems with some symbols (product, sum, integral, greater than, less then, equal). You also will see that the font size can't change (tiny, small, large, Large, LARGE) The admin wrote that the updated the kernel but they did not change anything with the LaTeX installation.
            Hide
            krause Ralf Krause added a comment -

            I tested the setting but they did not work.

            \usepackage[latin1]{inputenc}
            \usepackage{amsmath}
            \usepackage{accfonts}
            \RequirePackage{amsmath,amssymb,latexsym}

            Show
            krause Ralf Krause added a comment - I tested the setting but they did not work. \usepackage[latin1]{inputenc} \usepackage{amsmath} \usepackage{accfonts} \RequirePackage{amsmath,amssymb,latexsym}
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks Ralf,

            Can you check which tex-fonts are installed on server? Also, can you try delete Path to Latex (make field empty) and check if it works.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks Ralf, Can you check which tex-fonts are installed on server? Also, can you try delete Path to Latex (make field empty) and check if it works.
            Hide
            krause Ralf Krause added a comment -

            What should I do to see the fonts? I don't have a terminal account for the servers. I have to ask the admins at the providers.

            I deleted the path in the edumoodle.at platform. The Moodle platform seems to use MimeTeX. This works fine but the formulas get the gif format.

            I also deleted the path in fobi.schulon.org. No formula is generated now .... I think that they did not configure MimeTeX as executable.

            The platform www.moodletreff.de works with MimeTeX because the provider could not get the TeX filter working properly with latex. From this point I am writing about the problems with the TeX filter in the tracker.

            Show
            krause Ralf Krause added a comment - What should I do to see the fonts? I don't have a terminal account for the servers. I have to ask the admins at the providers. I deleted the path in the edumoodle.at platform. The Moodle platform seems to use MimeTeX. This works fine but the formulas get the gif format. I also deleted the path in fobi.schulon.org. No formula is generated now .... I think that they did not configure MimeTeX as executable. The platform www.moodletreff.de works with MimeTeX because the provider could not get the TeX filter working properly with latex. From this point I am writing about the problems with the TeX filter in the tracker.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks Ralf,

            If latex, dvips and convert is not available then mimeTeX will be used and it will create GIF images.
            I couldn't find much information about why this is happening, but seems to be related to fonts. On my dev machine, I observed the same issue (no font size/color was respected). Debugging further reviled font was giving problem, so I used another font (changed Tex preamble) and it worked.

            At this point, I am expecting something wrong with Latex conversion and would try some standalone latex code to check if it works fine on your server (to isolate the problem)

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks Ralf, If latex, dvips and convert is not available then mimeTeX will be used and it will create GIF images. I couldn't find much information about why this is happening, but seems to be related to fonts. On my dev machine, I observed the same issue (no font size/color was respected). Debugging further reviled font was giving problem, so I used another font (changed Tex preamble) and it worked. At this point, I am expecting something wrong with Latex conversion and would try some standalone latex code to check if it works fine on your server (to isolate the problem)
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Dropping security tag to add Marc

            Show
            rajeshtaneja Rajesh Taneja added a comment - Dropping security tag to add Marc
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Hello Marc,

            Can you please help us on this issue? What do you think, about adding --halt-on-error switch ?

            Show
            rajeshtaneja Rajesh Taneja added a comment - Hello Marc, Can you please help us on this issue? What do you think, about adding --halt-on-error switch ?
            Hide
            net-buoy Marc Grober added a comment -

            Well, you invited me, lol, so I am going to take a minute to do some recapping and hypothecating that is likely old hat for everyone, but I think necessary nonetheless....

            First off, follow along a bit as I talk about a rather arbitrary continuum of kludge, patch, and redesign of a core filter.

            As far as a kludge, bottom line is whether the kludge is the most economical resolution for a rare case to get by. Most are more than happy to decouple dragmath and add mathjax to head in additionalhtml - so the truth is, there may be only two reason to even address the underpinnings of the concern: a) for those who really need the the incremental utility provided by TexLive over Mathjax, and b) because beating a dead horse is always a good idea (and while I am being a bit tongue in cheek, as my discussion below suggests, not by that much....) Of course, what makes the interest in tweaking anything less inviting is the fact that TexLive is intended to be installed by the end user - that is to say that any user on a shared host need not rely on shared binaries..... and of course, any user on a shared system must, be definition, be able to make use of the MathJax CDN, though it is also easy enough to do a local install of MathJax on one's own server share. So, even at the beginning of our analysis it becomes dubious as to whether tweaking Moodle code is worthwhile.

            But now we move to the patch, and arguendo, the basis for the patch is that there is a proven issue that impacts a clear portion of the population for whom a quick code change will resolve their problems. [yes, I admitted my distinctions as to kludge, patch and redesign are somewhat arbitrary and inaccurate, but I am not doing this as a basis to build the trade towers, but as a tool for posing the questions I am musing, and I think they serve for such purposes] But in the case of a patch we are typically agreeing that there is indeed a code problem, and, as well, that a patch is a better way to address the problem, than recommending other solutions (a local install of TexLive, for example.) I have yet to be convinced that there is a security hole, especially in light of the most recent suggestion that the problem may only appear when there is a font issue. This situation begs the question first noted above, and that is whether the claimed hang is in Moodle php or is in fact an issue in the underlying technology (the Tex or other binaries – I know that there have been some bugs in Tex resulting from missing items though I am not aware of any current bugs, but that makes no difference as who knows what version of Tex any user may be employing?) And is that the real problem here, i.e. relying on a distro of binaries for which you are not testing provenance nor completeness of distribution? Is it better to fiddle the Moodle php code, or simply try to intelligently address the nature of the distro that is being installed, which is in fact most intelligently done by having the user do their own TexLive install, which leads us to redesign questions.

            A patch always poses the question of whether a redesign is necessary or appropriate, and that is indeed th question when we are talking about a filter that needs major reworking (I was going to do some of that but abandoned the work for a number of reason we don't need to rehash here.) The Moodle TeX filter expects certain performance from three binaries, two only of which are to be found in a Tex distribution, nor are all Tex distributions created equally. If the filter borks it falls back to mimetex, which was perhaps the best option at the time but even John argues that it is no longer. Among other things this means that Moodle will in a sense aggregate the vulnerabilities of its component parts. In this case the question that one arguably must ask first is whether the SA is concerned over a Tex security vulnerability, a Moodle vulnerability in calling the tex binary, or a Moodle configuration option, and whether arguably all of the above cold be resolved with a modern filter.

            Sorry if some old hands get their noses out of joint, but the Tex filter is long past its prime - simply requiring that anyone who wants to use Tex has to have convert installed is about as ridiculous as all get out considering that convert has not been necessary for almost a decade, lol. What is the deal? Perhaps its that no one wants to fight with core devs who have stated that Math is not a core concern of Moodle, lol. Whatever the reason, the filter has seen better days and arguably its sole purpose is to provide options for those, as I argued above, who insist on a full installation of Tex because, for example, they want access to polynom But anyone actually needing this install is going to certainly be able to do a local install of TexLive, and an effective redesign of a Tex filter would allow Moodle to implement meaningful fallback for the 21st Century (and avoid ancient artifacts, like requiring convert) while avoiding the third party disputes between webhosts and Moodle users (which, unlike the current case, often involve folk who are WAYYYYY out of their depth.)

            That all being said, what do I suggest one take away from this ramble? DO what you wish with respect to the Tex filter as almost anyone asking about Tex is being told to abandon that ship anyway! For those who do need a full install should tweaks be made to address such a situation as described above? I think not, as if the problem is with Tex, then it is a Tex problem which they need to fix, and beyond that there are ample solutions that minimize the sources of error. Indeed, I would rather see an install script in Moodle that actually did a TexLive install for the user and then configured the filter based on that install, than the current situation. This response is perhaps even underlined by the tentative results regarding fonts. Is it Moodle's responsibility to be the guarantors of the webhosts Tex distro? I think that is someplace we do not need to tread.

            But, should it be demonstrable that this error could be forced, i.e. that there is truly an exploit possible if a full proper install is in place, that is another issue altogether. But so far, what we are talking about is a third party system continuing to try and source a resource that it can't find and I don't know that we have determined that there is no time out. As noted, not only can't we guarantee the distribution and its installation, we really can't guarantee the configuration. As a result I think we should not be mucking about with core unless we are going to fix things, and we are not going to fix things by tweaking this parameter. SHould it be determined that a specific configuration can cause an issue, we should flag that issue in the docs and perhaps provide a default preamble that explains the issue.

            For many, it is almost an Herculean task to get Tex running on a shared host. adding hidden gotchas like would be the worst - but, suggesting a flag to the binary is arguably acceptable as long as it remains only that.............

            I am afraid that I have been overly discursive and not helpful in the manner you had hoped, and for that you have my apologies, but I do hope I have been of some assistance.

            Show
            net-buoy Marc Grober added a comment - Well, you invited me, lol, so I am going to take a minute to do some recapping and hypothecating that is likely old hat for everyone, but I think necessary nonetheless.... First off, follow along a bit as I talk about a rather arbitrary continuum of kludge, patch, and redesign of a core filter. As far as a kludge, bottom line is whether the kludge is the most economical resolution for a rare case to get by. Most are more than happy to decouple dragmath and add mathjax to head in additionalhtml - so the truth is, there may be only two reason to even address the underpinnings of the concern: a) for those who really need the the incremental utility provided by TexLive over Mathjax, and b) because beating a dead horse is always a good idea (and while I am being a bit tongue in cheek, as my discussion below suggests, not by that much....) Of course, what makes the interest in tweaking anything less inviting is the fact that TexLive is intended to be installed by the end user - that is to say that any user on a shared host need not rely on shared binaries..... and of course, any user on a shared system must, be definition, be able to make use of the MathJax CDN, though it is also easy enough to do a local install of MathJax on one's own server share. So, even at the beginning of our analysis it becomes dubious as to whether tweaking Moodle code is worthwhile. But now we move to the patch, and arguendo, the basis for the patch is that there is a proven issue that impacts a clear portion of the population for whom a quick code change will resolve their problems. [yes, I admitted my distinctions as to kludge, patch and redesign are somewhat arbitrary and inaccurate, but I am not doing this as a basis to build the trade towers, but as a tool for posing the questions I am musing, and I think they serve for such purposes] But in the case of a patch we are typically agreeing that there is indeed a code problem, and, as well, that a patch is a better way to address the problem, than recommending other solutions (a local install of TexLive, for example.) I have yet to be convinced that there is a security hole, especially in light of the most recent suggestion that the problem may only appear when there is a font issue. This situation begs the question first noted above, and that is whether the claimed hang is in Moodle php or is in fact an issue in the underlying technology (the Tex or other binaries – I know that there have been some bugs in Tex resulting from missing items though I am not aware of any current bugs, but that makes no difference as who knows what version of Tex any user may be employing?) And is that the real problem here, i.e. relying on a distro of binaries for which you are not testing provenance nor completeness of distribution? Is it better to fiddle the Moodle php code, or simply try to intelligently address the nature of the distro that is being installed, which is in fact most intelligently done by having the user do their own TexLive install, which leads us to redesign questions. A patch always poses the question of whether a redesign is necessary or appropriate, and that is indeed th question when we are talking about a filter that needs major reworking (I was going to do some of that but abandoned the work for a number of reason we don't need to rehash here.) The Moodle TeX filter expects certain performance from three binaries, two only of which are to be found in a Tex distribution, nor are all Tex distributions created equally. If the filter borks it falls back to mimetex, which was perhaps the best option at the time but even John argues that it is no longer. Among other things this means that Moodle will in a sense aggregate the vulnerabilities of its component parts. In this case the question that one arguably must ask first is whether the SA is concerned over a Tex security vulnerability, a Moodle vulnerability in calling the tex binary, or a Moodle configuration option, and whether arguably all of the above cold be resolved with a modern filter. Sorry if some old hands get their noses out of joint, but the Tex filter is long past its prime - simply requiring that anyone who wants to use Tex has to have convert installed is about as ridiculous as all get out considering that convert has not been necessary for almost a decade, lol. What is the deal? Perhaps its that no one wants to fight with core devs who have stated that Math is not a core concern of Moodle, lol. Whatever the reason, the filter has seen better days and arguably its sole purpose is to provide options for those, as I argued above, who insist on a full installation of Tex because, for example, they want access to polynom But anyone actually needing this install is going to certainly be able to do a local install of TexLive, and an effective redesign of a Tex filter would allow Moodle to implement meaningful fallback for the 21st Century (and avoid ancient artifacts, like requiring convert) while avoiding the third party disputes between webhosts and Moodle users (which, unlike the current case, often involve folk who are WAYYYYY out of their depth.) That all being said, what do I suggest one take away from this ramble? DO what you wish with respect to the Tex filter as almost anyone asking about Tex is being told to abandon that ship anyway! For those who do need a full install should tweaks be made to address such a situation as described above? I think not, as if the problem is with Tex, then it is a Tex problem which they need to fix, and beyond that there are ample solutions that minimize the sources of error. Indeed, I would rather see an install script in Moodle that actually did a TexLive install for the user and then configured the filter based on that install, than the current situation. This response is perhaps even underlined by the tentative results regarding fonts. Is it Moodle's responsibility to be the guarantors of the webhosts Tex distro? I think that is someplace we do not need to tread. But, should it be demonstrable that this error could be forced, i.e. that there is truly an exploit possible if a full proper install is in place, that is another issue altogether. But so far, what we are talking about is a third party system continuing to try and source a resource that it can't find and I don't know that we have determined that there is no time out. As noted, not only can't we guarantee the distribution and its installation, we really can't guarantee the configuration. As a result I think we should not be mucking about with core unless we are going to fix things, and we are not going to fix things by tweaking this parameter. SHould it be determined that a specific configuration can cause an issue, we should flag that issue in the docs and perhaps provide a default preamble that explains the issue. For many, it is almost an Herculean task to get Tex running on a shared host. adding hidden gotchas like would be the worst - but, suggesting a flag to the binary is arguably acceptable as long as it remains only that............. I am afraid that I have been overly discursive and not helpful in the manner you had hoped, and for that you have my apologies, but I do hope I have been of some assistance.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks for the feedback Marc,
            AFAIK there is no similar issue reported, and I agree tex install is local to user.

            Sorry Ralf, but it seems to be issue specific to your server and not sure adding --halt-on-error will solve this issue. If you still think it will be helpful, then I can create patch and try get some feedback from integrators.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks for the feedback Marc, AFAIK there is no similar issue reported, and I agree tex install is local to user. Sorry Ralf, but it seems to be issue specific to your server and not sure adding --halt-on-error will solve this issue. If you still think it will be helpful, then I can create patch and try get some feedback from integrators.
            Hide
            krause Ralf Krause added a comment -

            Thank you very much for the feedback to Marc also from my side.

            The three servers that I 'm using are provided by professional hosters and I hope they know how to install the basic software. I can't say where the problems come from. Perhaps everything is okay with the Moodle TeX filter but we need a better installation tutorial for the base TeX installation.

            Rajesh, please try to add the parameter. Perhaps it would be a good choice to get the parameter --halt-on-error as an optional setting in the TeX filter. If everything is okay with the tex installation it should be no problem if the option is activated or deactivated. But if there are problems it would be a good chance to minimize the server load and to debug the problem.

            Ralf

            Show
            krause Ralf Krause added a comment - Thank you very much for the feedback to Marc also from my side. The three servers that I 'm using are provided by professional hosters and I hope they know how to install the basic software. I can't say where the problems come from. Perhaps everything is okay with the Moodle TeX filter but we need a better installation tutorial for the base TeX installation. Rajesh, please try to add the parameter. Perhaps it would be a good choice to get the parameter --halt-on-error as an optional setting in the TeX filter. If everything is okay with the tex installation it should be no problem if the option is activated or deactivated. But if there are problems it would be a good chance to minimize the server load and to debug the problem. Ralf
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            Thanks Ralf,

            I have added the switch, and done preliminary testing to make sure all works as expected.
            It will be helpful for testing, if you can please send some tex strings which will fail and stop Latex execution.

            Show
            rajeshtaneja Rajesh Taneja added a comment - Thanks Ralf, I have added the switch, and done preliminary testing to make sure all works as expected. It will be helpful for testing, if you can please send some tex strings which will fail and stop Latex execution.
            Hide
            net-buoy Marc Grober added a comment -

            LOL, yes, the TeX filter is a bit of a mess, Raif...... but you have a variety of options, the most promising being your own install of TexLive, which is arguably easier than installing Moodle, though as I mention, the use of MathJax, jsxgraph, etc via additionalhtml is very inviting. But if you did do your own install of TexLive you could also help determine what if any of the issues you are encountering could be eliminated in such a fashion, as well as provide a first draft of a texlive non root install for Moodle users

            Show
            net-buoy Marc Grober added a comment - LOL, yes, the TeX filter is a bit of a mess, Raif...... but you have a variety of options, the most promising being your own install of TexLive, which is arguably easier than installing Moodle, though as I mention, the use of MathJax, jsxgraph, etc via additionalhtml is very inviting. But if you did do your own install of TexLive you could also help determine what if any of the issues you are encountering could be eliminated in such a fashion, as well as provide a first draft of a texlive non root install for Moodle users
            Hide
            salvetore Michael de Raadt added a comment -

            Looks good to me.

            Show
            salvetore Michael de Raadt added a comment - Looks good to me.
            Hide
            poltawski Dan Poltawski added a comment -

            The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

            TIA and ciao

            Show
            poltawski Dan Poltawski added a comment - The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week. TIA and ciao
            Hide
            poltawski Dan Poltawski added a comment -

            Urm.

            Are you treating things too literally here... "-halt-on-error parameter" I think should just by "-halt-on-error" because it is the halt on error paraemter.

            Show
            poltawski Dan Poltawski added a comment - Urm. Are you treating things too literally here... "- halt-on-error parameter" I think should just by " -halt-on-error" because it is the halt on error paraemter.
            Hide
            rajeshtaneja Rajesh Taneja added a comment -

            grrr..

            Thanks Dan, updated the patch.

            Show
            rajeshtaneja Rajesh Taneja added a comment - grrr.. Thanks Dan, updated the patch.
            Hide
            poltawski Dan Poltawski added a comment -

            Thanks Raj, i've integrated this now.

            Show
            poltawski Dan Poltawski added a comment - Thanks Raj, i've integrated this now.
            Hide
            abgreeve Adrian Greeve added a comment -

            Installation of TeX was simple.
            Thanks for the easy to follow instructions.
            I had no problem displaying a label with TeX notation.
            Test passed

            Show
            abgreeve Adrian Greeve added a comment - Installation of TeX was simple. Thanks for the easy to follow instructions. I had no problem displaying a label with TeX notation. Test passed
            Hide
            poltawski Dan Poltawski added a comment -

            *Notice*: Undefined variable: friendlyintegrator in /Users/danp/git/tokenintegrationthanks.php on line 26

            Congratulations

            {tracker.user.name}

            !

            You've made into Moodle

            {tracker.fixversion-1}

            +

            I would like to personally thank you for this contribution on behalf of all Moodle users throughout the world.

            cheers!

            {tracker.friendlyintegrator}
            Show
            poltawski Dan Poltawski added a comment - * Notice *: Undefined variable: friendlyintegrator in /Users/danp/git/tokenintegrationthanks.php on line 26 Congratulations {tracker.user.name} ! You've made into Moodle {tracker.fixversion-1} + I would like to personally thank you for this contribution on behalf of all Moodle users throughout the world. cheers! {tracker.friendlyintegrator}
            Hide
            krause Ralf Krause added a comment -

            Hi,
            will we get this patch also for Moodle 2.2 and Moodle 2.3?
            Ralf

            Show
            krause Ralf Krause added a comment - Hi, will we get this patch also for Moodle 2.2 and Moodle 2.3? Ralf
            Hide
            vadimon Vadim Dvorovenko added a comment -

            Hello everyone. This issue is one of the most terrible mistake around moodle that i've ever seen and should be rolled back. It causes regression. Let me explain.
            I have recently upgraded one large site from moodle 1.9 to moodle 2.6. This site contains huge amount of math formulas (mode than 10 000). After upgrade a lot of formulas became unusable without any visible reason. I've tried using different preambles, different latex packages but i still had some formulas that looked perfect in 1.9 and was not rendered in 2.6. You should understand, that we don't write tex directly - during last 5 years all formulas were converted from different places with different convertors - texadie, mathtype, word2tex, tinymce plugin, and there's none user except for me in all the university, that can read and write latex code directly and can find a mistakes in latex formulas.
            When latex did not halts on error, it just skips unknown commands and symbols and tries to show formula, so formulas with little problems still looks good. Some examples to proove, that it is not the regular user solvable problem, but admin's problem.
            If you have big formula (for example 5 or more line) edited with tinymce, you cannot see unicode unbreakable space symbol (\u00A0), and will have no way to determine if the formula contains it, but tinymce can insert it on his own, for example, when you copy/paste from word some formulas, that was converted from mathtype. Latex (with default preamble) does know this unicode symbol and will not show such formula.
            In russian math tradition we designate tan and cot functions as tg and ctg. To make this functions look as expected we use legal latex \operatorname

            {tg} construction (not we, of course, but mathtype converter). But 'name' is forbidden word for mathtype filter and this become \operatorforbiddenkeyword{tg}

            . As far as latex does not know such command, it haltes on error and does not render formula. Without --halt-on-error the bad command is skipped and this example looks like italics 'tg', wich is nearly expected roman 'tg' and leaves formula readable.
            So, some formulas, imported or upgraded from older moodle will become unusable in newer moodle and that's regression.

            Show
            vadimon Vadim Dvorovenko added a comment - Hello everyone. This issue is one of the most terrible mistake around moodle that i've ever seen and should be rolled back. It causes regression. Let me explain. I have recently upgraded one large site from moodle 1.9 to moodle 2.6. This site contains huge amount of math formulas (mode than 10 000). After upgrade a lot of formulas became unusable without any visible reason. I've tried using different preambles, different latex packages but i still had some formulas that looked perfect in 1.9 and was not rendered in 2.6. You should understand, that we don't write tex directly - during last 5 years all formulas were converted from different places with different convertors - texadie, mathtype, word2tex, tinymce plugin, and there's none user except for me in all the university, that can read and write latex code directly and can find a mistakes in latex formulas. When latex did not halts on error, it just skips unknown commands and symbols and tries to show formula, so formulas with little problems still looks good. Some examples to proove, that it is not the regular user solvable problem, but admin's problem. If you have big formula (for example 5 or more line) edited with tinymce, you cannot see unicode unbreakable space symbol (\u00A0), and will have no way to determine if the formula contains it, but tinymce can insert it on his own, for example, when you copy/paste from word some formulas, that was converted from mathtype. Latex (with default preamble) does know this unicode symbol and will not show such formula. In russian math tradition we designate tan and cot functions as tg and ctg. To make this functions look as expected we use legal latex \operatorname {tg} construction (not we, of course, but mathtype converter). But 'name' is forbidden word for mathtype filter and this become \operatorforbiddenkeyword{tg} . As far as latex does not know such command, it haltes on error and does not render formula. Without --halt-on-error the bad command is skipped and this example looks like italics 'tg', wich is nearly expected roman 'tg' and leaves formula readable. So, some formulas, imported or upgraded from older moodle will become unusable in newer moodle and that's regression.
            Hide
            poltawski Dan Poltawski added a comment -

            Hi Vadim,

            Please can you create a new issue for this with some detailed replication steps, as this one will not be reopened.

            thanks,
            Dan

            Show
            poltawski Dan Poltawski added a comment - Hi Vadim, Please can you create a new issue for this with some detailed replication steps, as this one will not be reopened. thanks, Dan
            Hide
            vadimon Vadim Dvorovenko added a comment -

            Oh, i've forgotten to make link. MDL-44400

            Show
            vadimon Vadim Dvorovenko added a comment - Oh, i've forgotten to make link. MDL-44400

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  3/Dec/12