Add-ons
  1. Add-ons
  2. CONTRIB-2368

SHEBanG - An alternate enrol module/plugin for SunGard Banner/LMB

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.9.9
    • Fix Version/s: 1.9.10
    • Component/s: Enrol: Shebang
    • Labels:
      None
    • Affected Branches:
      MOODLE_19_STABLE
    • Fixed Branches:
      MOODLE_19_STABLE
    • Rank:
      33997

      Description

      The enrol_shebang module is used to import Banner/LMB course, person, and enrollment messages into Moodle. It is still in beta and being tested. The README.txt file contents are pasted below.

      README.txt for SHEBanG enrolment plugin/module

      DISCLAIMER AND LICENSING
      ------------------------
      SHEBanG enrolment plugin/module for SunGard HE Banner(r) data import

      This program is free software: you can redistribute it and/or modify
      it under the terms of the GNU General Public License as published by
      the Free Software Foundation, either version 3 of the License, or (at
      your option) any later version.

      This program is distributed in the hope that it will be useful, but
      WITHOUT ANY WARRANTY; without even the implied warranty of
      MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
      General Public License for more details.

      You should have received a copy of the GNU General Public License
      along with this program. If not, see <http://www.gnu.org/licenses/>.

      GENERAL INFORMATION
      -------------------
      The SHEBanG enrolment plugin is designed to import Luminis Message
      Broker (LMB) messages containing data sent from a SunGard HE Banner
      installation. The messages are POSTed by LMB to a specified URL;
      that URL should end up resolving to enrol/shebang/secure/post.php
      either by means of direct address, symbolic link, or Apache server
      configuration (e.g. alias). An alternate method of import is the
      file upload feature.

      There already is an enrolment plugin, LMB, developed by Eric Merrill
      <merrill@oakland.edu> that processes Banner data, and has many more
      configuration options and features. We have been using this plugin
      for several years at Appalachian State with only a few basic alter-
      ations.

      So why write a new one? Why reinvent? There were a few reasons.

      In our particular installtion of Banner/LMB, we kept getting several
      essentially identical messages which resulted in duplicate rows in
      one of the staging tables. Normally not a big problem, but eventually
      the staging table becomes bloated with duplicates. We also wanted to
      log each of the XML messages received, but not in the database.

      Putting them in a text file is more practical for us. Messages sent
      by the LMB are logged to the file-system in the Moodle data directory
      rather than the database; this message log file is locked while the
      message is being processed in order to serialize message processing.

      In the case where additional data from the XML message was needed, a
      regular expression had to be added--most often this is simple enough
      given an existing pattern from which to start, but in some instances
      where the message format changes ever so slightly (e.g. recstatus),
      it makes more sense to use DOMXPath queries to extract message data.

      XMLParser is used to break up the input, whether from file or posted
      XML, into the principle elements we want (<group>, <person>, and
      <membership>), and from that point use the DOMDocument and DOMXPath
      to query for the needed values.

      Some other differences:

      Messages imported from a file are not logged since there's already a
      source file;

      A cross-list course parent is created only when the first <membership>
      message for that cross-list arrives, and then it will be created by
      making a copy of the child course.

      The Moodle user's idnumber column is populated with the SCTID/Banner
      userid, rather than the Luminis sourcedid/id.

      The only cron activity is the monitor for last message arrival time.
      Some Moodle sites may have a very frequent cron activity (5-10 min)
      so it didn't seem practical to put dir/file check tasks in a common
      script--rather use a dedicated cron task, if needed at all, for the
      Banner extract/batch-file imports.

      INSTALLATION
      ------------
      Place the shebang directory in the site's enrol directory. Access the
      notifications page (Admin block->Notifications) to initiate database
      setup.

      Enable and configure the plugin (Courses->Enrolments->SHEBanG[Edit]).

      Configure your Banner/LMB to POST messages to the URL corresponding
      to the enrol/shebang/secure/post.php file, for example:

      http://moodle.someuniversity.edu/enrol/shebang/secure/post.php

      • * * * * * * * * I M P O R T A N T * * * * * * * * *

      THIS URL NEEDS TO BE SECURED IN SOME FASHION TO PREVENT UNAUTHORIZED
      USERS FROM INJECTING ERRANT DATA INTO YOUR DATABASE. THE METHODS FOR
      SECURING THIS URL INCLUDE, BUT ARE NOT LIMITED TO, APACHE HTTP AUTH
      CONFIGURED-IDEALLY WITH SSL-AT THE SERVER/VHOST/DIRECTORY LEVEL OR
      WITH AN .htaccess FILE. ANOTHER OPTION, IS TO USE THE AUTHENTICATION
      FEATURE IN THE MODULE, BUT AGAIN IT IS STRONGLY RECOMMENDED TO USE A
      SECURE SOCKET CONNECTION IN ADDITION TO HTTP BASIC OR DIGEST AUTHEN-
      TICATION SINCE EITHER THE USERID/PWD INFORMATION WILL BE EXPOSED
      WHEN SENT IN CLEAR-TEXT (BASIC) OR THE (DIGEST) HASH WILL BE EXPOSED.

      There is no Moodle authentication protection in the post.php since it
      is intended that only Banner LMB will be accessing this URL and a
      separate userid/password will be configured by the Moodle server admin
      and shared with the Luminis admins.

      CONSIDERATIONS FOR BLOCKING ISSUES
      ----------------------------------
      Because SHEBanG serializes message processing using a file lock, it
      is possible, in some cases likely, that a blocking bottleneck will
      take place when a large number of messages are sent by Banner/LMB at
      or near the same time. To mitigate the risk of the all the Apache
      worker processes being occupied and blocked, and thus freezing out
      the application end-users, consider setting up another Apache daemon
      to handle LMB messages exclusively on an alternate port such as 8080.
      In this way you can tailor the configured number of initial, max-min
      spare worker processes, etc., and if these processes should become
      consumed, the end-users will not be impacted.

      HOW ENTITIES ARE ASSOICATED AND REFERENCED
      ------------------------------------------
      Courses:
      A staging record in the [mdl_enrol_shebang_section] table is inserted
      or updated, using the <sourcedid><id> value (the CRN) as the alternate
      key. This same value is placed in the [mdl_course](idnumber) column so
      that table can be queried directory. The course's (idnumber) value can
      not be changed after that or the association will be lost.

      Users:
      A staging record in the [mdl_enrol_shebang_person] table is inserted
      or updated, using the <sourcedid><id> value (the Luminis Id for that
      Banner identity, distinct from the Banner identity value). Susbsequent
      <membership> messages for this person will use this <sourcedid><id> so
      we need to use this as an alternate identifying key value. A <person>
      message also contains the <userid useridtype="SCTID"> value, the
      Banner identity, and it's this value that is placed in the [mdl_user]
      (idnumber) column.

      So, upon arrival of a <person> message, and after the staging record
      is handled, the [mdl_user] table is queried on the (idnumber) column
      for the SCTID value--if no record is found, then an attempt is made to
      insert one, otherwise the found record is updated. Because other
      columns in the [mdl_user] table are unique (enforced either by the
      application or the database), a collision on email or username by
      prevent the insert. In such cases, manual inspection of the existing
      user account should be done to determine if it can be deleted (no
      access, no enrolments, etc.) or if it should be associated with the
      corresponding [mdl_enrol_shebang_person] row by placing the appro-
      priate Banner identity value in the user's (idnumber) column.

      Until any such collision is resolved, course enrollments for that
      person/user will fail since no association exists between the person
      entity and the user entity.

        Issue Links

          Activity

          Hide
          Fred Woolard added a comment -

          Use this for evaluation, please – updated README and lang files

          Show
          Fred Woolard added a comment - Use this for evaluation, please – updated README and lang files
          Hide
          Anthony Borrow added a comment -

          Many thanks for sharing this contributed code with the Moodle community. I have begun reviewing your contribution and let you know any comments, questions, suggestion, or concerns I may have. I do my best to ensure that the file structure is consistent with the norms for Moodle contributed code so that we can avoid having to move files around which is a bit challenging with CVS. My apologies for the delay as my "paying job" has been keeping me busier than usual. Peace - Anthony

          Show
          Anthony Borrow added a comment - Many thanks for sharing this contributed code with the Moodle community. I have begun reviewing your contribution and let you know any comments, questions, suggestion, or concerns I may have. I do my best to ensure that the file structure is consistent with the norms for Moodle contributed code so that we can avoid having to move files around which is a bit challenging with CVS. My apologies for the delay as my "paying job" has been keeping me busier than usual. Peace - Anthony
          Hide
          Anthony Borrow added a comment -

          I have added the code to Moodle's CVS server. In order to have write access, please request it at http://moodle.org/cvs/ (if you have not already done so). Let me know when you do so that I can review it and get that approved for your promptly. Keep in mind, that it is recommended that all CVS commit comments should contain a Moodle tracker number as the tracker and CVS server are setup to create autolinks between them which goes a long way toward providing good documentation.

          Show
          Anthony Borrow added a comment - I have added the code to Moodle's CVS server. In order to have write access, please request it at http://moodle.org/cvs/ (if you have not already done so). Let me know when you do so that I can review it and get that approved for your promptly. Keep in mind, that it is recommended that all CVS commit comments should contain a Moodle tracker number as the tracker and CVS server are setup to create autolinks between them which goes a long way toward providing good documentation.
          Hide
          Anthony Borrow added a comment -

          I am moving this issue to the newly created Tracker component for the code which you have contributed. I have assigned you as the component lead which means that you will automatically be assigned any issues created for this component.

          Show
          Anthony Borrow added a comment - I am moving this issue to the newly created Tracker component for the code which you have contributed. I have assigned you as the component lead which means that you will automatically be assigned any issues created for this component.
          Hide
          Anthony Borrow added a comment -

          At this point, I like to encourage reviewing http://docs.moodle.org/en/Development:Guidelines_for_contributed_code as the guidelines provide additional detail for sharing news of this plugin with the Moodle community in the Moodle.org Contributed modules and plugins forum (http://moodle.org/mod/forum/view.php?id=44), creating a documentation page in Moodle Docs (http://docs.moodle.org/), and adding an entry to the Modules and Plugins database (http://moodle.org/mod/data/view.php?id=6009) where users look to find plugins. These steps help to ensure that users can find and install your code in a consistent manner. In order for the Modules and plugins database entry to be visible to the public, it needs to be approved by a site moderator. Just let me know the URL of the entry once you create and I will make sure to review it promptly. Also, if at any point you have to modify the Modules and plugins entry it will need to be re-approved so a quick Moodle message or comment here in the tracker will make sure it get reviewed and re-approved as quickly as possible. I am going to resolve this issue since the code is in CVS, the tracker component is created, etc. Do not hesitate to let me know how I can best be supportive of your efforts. Peace - Anthony

          Show
          Anthony Borrow added a comment - At this point, I like to encourage reviewing http://docs.moodle.org/en/Development:Guidelines_for_contributed_code as the guidelines provide additional detail for sharing news of this plugin with the Moodle community in the Moodle.org Contributed modules and plugins forum ( http://moodle.org/mod/forum/view.php?id=44 ), creating a documentation page in Moodle Docs ( http://docs.moodle.org/ ), and adding an entry to the Modules and Plugins database ( http://moodle.org/mod/data/view.php?id=6009 ) where users look to find plugins. These steps help to ensure that users can find and install your code in a consistent manner. In order for the Modules and plugins database entry to be visible to the public, it needs to be approved by a site moderator. Just let me know the URL of the entry once you create and I will make sure to review it promptly. Also, if at any point you have to modify the Modules and plugins entry it will need to be re-approved so a quick Moodle message or comment here in the tracker will make sure it get reviewed and re-approved as quickly as possible. I am going to resolve this issue since the code is in CVS, the tracker component is created, etc. Do not hesitate to let me know how I can best be supportive of your efforts. Peace - Anthony
          Hide
          Fred Woolard added a comment -

          I have updated the CVS directories for which I have requested write access to include /contrib/plugins/enrol/shebang

          Show
          Fred Woolard added a comment - I have updated the CVS directories for which I have requested write access to include /contrib/plugins/enrol/shebang
          Hide
          Anthony Borrow added a comment -

          Fred - There is currently a bug on moodle.org/cvs/ that does not allow me to change the privileges. Let me see if I can get someone to give that some attention today. Peace - Anthony

          Show
          Anthony Borrow added a comment - Fred - There is currently a bug on moodle.org/cvs/ that does not allow me to change the privileges. Let me see if I can get someone to give that some attention today. Peace - Anthony
          Hide
          Fred Woolard added a comment -

          Anthony - I've gone ahead and created the corresponding Moodle Docs page http://docs.moodle.org/en/Development:SHEBanG_plugin, and the M&P db entry http://moodle.org/mod/data/view.php?d=13&rid=4264. Thanks very much for the help with this. Regards - FW.

          Show
          Fred Woolard added a comment - Anthony - I've gone ahead and created the corresponding Moodle Docs page http://docs.moodle.org/en/Development:SHEBanG_plugin , and the M&P db entry http://moodle.org/mod/data/view.php?d=13&rid=4264 . Thanks very much for the help with this. Regards - FW.
          Hide
          Anthony Borrow added a comment -

          I've approved the M&P entry so that is now visible to the public. Let me know if you want a 1.9 branch of the code when you want to use HEAD to start working on 2.0 Until then it is probably best just to have the one version to maintain. Peace - Anthony

          Show
          Anthony Borrow added a comment - I've approved the M&P entry so that is now visible to the public. Let me know if you want a 1.9 branch of the code when you want to use HEAD to start working on 2.0 Until then it is probably best just to have the one version to maintain. Peace - Anthony
          Hide
          Fred Woolard added a comment -

          It appears that the M&P entry is no longer approved, and no longer visible to the public.

          Show
          Fred Woolard added a comment - It appears that the M&P entry is no longer approved, and no longer visible to the public.
          Hide
          Anthony Borrow added a comment -

          Maybe I just imagined that I had approved it but have actually done so now. If you edit the entry it will need to be re-approved but that is just a formality and I'll do it quickly. If you could, just send me the URL to the page in a Moodle message when it needs to be re-approved and I'll get it taken care of right away. Peace - Anthony

          Show
          Anthony Borrow added a comment - Maybe I just imagined that I had approved it but have actually done so now. If you edit the entry it will need to be re-approved but that is just a formality and I'll do it quickly. If you could, just send me the URL to the page in a Moodle message when it needs to be re-approved and I'll get it taken care of right away. Peace - Anthony
          Hide
          Fred Woolard added a comment -

          Anthony,

          I've spent a little time preparing this enrol plugin for Moodle 2.0, and just need to finish the config interface changes before preliminary testing. Since it is a significant departure from the 1.9 code, it would be handy if you were to go ahead and set up a branch for MOODLE_19_STABLE for this project.

          Regards,
          FW

          Show
          Fred Woolard added a comment - Anthony, I've spent a little time preparing this enrol plugin for Moodle 2.0, and just need to finish the config interface changes before preliminary testing. Since it is a significant departure from the 1.9 code, it would be handy if you were to go ahead and set up a branch for MOODLE_19_STABLE for this project. Regards, FW
          Hide
          Anthony Borrow added a comment -

          Fred - I have updated your CVS access so that should be working now. I have also created the 1.9 branch so that you can begin using HEAD for 2.0. Let me know if you have any questions in terms of the links for the download files and of course if you update the Modules and Plugins database entry just let me know the URL and I'll get it re-approved. Peace - Anthony

          Show
          Anthony Borrow added a comment - Fred - I have updated your CVS access so that should be working now. I have also created the 1.9 branch so that you can begin using HEAD for 2.0. Let me know if you have any questions in terms of the links for the download files and of course if you update the Modules and Plugins database entry just let me know the URL and I'll get it re-approved. Peace - Anthony
          Hide
          Anthony Borrow added a comment -

          Closing all of my resolved issues. Peace - Anthony

          Show
          Anthony Borrow added a comment - Closing all of my resolved issues. Peace - Anthony

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development