History | Log In     View a printable version of the current page.  
We are currently focused especially on Moodle 2.0, Moodle 1.9.x bugs and Moodle 1.9.x testing.    Confused? Lost? Please read this introduction to the Tracker.
Issue Details (XML | Word | Printable)

Key: MDL-12886
Type: Bug Bug
Status: Open Open
Priority: Critical Critical
Assignee: Ludo ( Marc Alier)
Reporter: Martin Dougiamas
Votes: 7
Watchers: 19
Operations

If you were logged in you would be able to see more operations.
Moodle

Implement basic web service function set

Created: 09/Jan/08 04:51 PM   Updated: Tuesday 05:15 PM
Component/s: Lib, Networking
Affects Version/s: 1.9
Fix Version/s: 2.0

File Attachments: 1. File dfws_webservice.tgz (24 kb)
2. File dfws_webservice.tgz (21 kb)
3. XML File WSMoodleForGenDep.wsdl (63 kb)


Participants: Brian King, David Castro, Eloy Lafuente (stronk7), Frédéric Hoogstoel, Jerome Mouneyrac, Jordi Piguillem Poch, Ken DeVellis, Ludo ( Marc Alier), Martin Dougiamas, Matteo Bertazzo and Patrick Pollet
Security Level: None

Sub-Tasks  All   Open   
 Sub-Task Progress: 

 Description  « Hide
We need to implement the functions described here:

  http://docs.moodle.org/en/Development:Web_services

Most of the functions need to be implemented / integrated in core libraries, and the web services wrappers just call those.

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Martin Dougiamas - 24/Jan/08 03:18 PM
All of these subtasks need specifications and discussion before development

Ludo ( Marc Alier) - 21/Feb/08 04:30 PM
We have I implemented some functions as first prototype.
Howevere we need write access to the CVS, we lost it since the migration from sourceforge.
Users : granludo pigui tusefomal davcastro
thanks!
L.

Martin Dougiamas - 21/Feb/08 09:38 PM
All developers need to go here to reapply now: http://moodle.org/cvs (make sure you log in)

Eloy Lafuente (stronk7) - 22/Feb/08 02:35 AM
I've "accepted" request from developer "pigui" granting CVS access to "/contrib/plugins/mod/wiki" as requested.

Note that, in the request, "pigui" asked about extra-access to know where to store their WS-related code.... Martin?

Ciao :-)


Martin Dougiamas - 08/Apr/08 11:25 AM
Pigui, any progress on this? I can't see any and it's getting urgent.

Ludo ( Marc Alier) - 08/Apr/08 03:30 PM
Sorry for the delay... we are commiting the code today.

Frédéric Hoogstoel - 08/Apr/08 05:13 PM
Hi,

we have developped for two years web services for Moodle 1.8.2, under PHP5.
They are available on http://www.assembla.com/spaces/MoodleWSPourGenDep.
On this site, you'll find also a PHP client and a Java client to test these web services and see how to use them.

These web services have been developped during a project name Bricoles and the PhD Thesis in Computer Science and Learning Sciences of Pierre-André Caron. See http://noce.univ-lille1.fr/projets/bricoles/ for the project and http://tel.archives-ouvertes.fr/tel-00156376 for the thesis.
The goal of these web services is to help learning design using Model Driven Engineering.
We (NOCE research team) have developped a Model Driven Learning Engineering workshop, including two free tools :
- ModX that allows to build learning design metamodels and model
- GenDep that allows to deploy a learning setup model on a learning platform.
GenDep uses web services to communicate with the learning platform.
GenDep already exploits Moodle, Ganesha, Claroline and Wiki MST webservices to deploy learning setups.

ModX and GenDep are available at : http://noce.univ-lille1.fr/projets/ModX/index.php

We have presented our project at french-speaking Moodlemoot 2007 and we will present the evolution of our work at french-speaking Moodlemoot 2008.

We are currently studying how to articulate our web services with those described in this issue.

Frédéric Hoogstoel - 08/Apr/08 06:46 PM
WSDL description of web services for Moodle 1.8.2 developped by NOCE research team in Bircoles project.

Ludo ( Marc Alier) - 08/Apr/08 11:13 PM
Done... we have commited our work. We will try to work directy over the cvs when all of us can access it without problems.
There's a layer of php functions implementing the functionality, there's also a SOAP layer that implements calls to these php functions and finally there's a pyton client that connects to the soap ws.
Next week we will post more detailed documentation.
Ludo


Patrick Pollet - 12/Jun/08 05:56 PM
At the French MoodleMoot 2008, Frederic Hoogstoel, author of a SOAP WS with operations mostly aimed at creating "personnalized" courses from a learning profile determined outside Moodle (ie create/modify categories and courses, adding sections, forums, wikis...to courses and enroling/unenroling) , and me, current maintainer of the OK Web service implementation, that is more focused on "getting informations out of Moodle" (users, courses, activities, roles, grades...) have agreed to merge our two implementations that are really complementary.

Most of the operations he is currently supporting are not in mine and conversely.

We are really waiting for Ludo's implementation of " STANDARD list of CORE web service function" to start reducing our lower layer (server.class.php) that currently speaks (most of the time) to Moodle core API and in some rare cases do direct SQL calls ( such as when retrieving last activities in a course).

In the meantime we will focus on "merging" our high layer implementations that will finally propose about 80 operations, why names and parameters as per current OK WS 1.5.10 revision ( see wspp directory in CVS contrib/patches/ws )

More to come in future weeks.

If you can read french, our two communications at the Freench MM2K8 are here

http://moodlemoot2008.vet-nantes.fr/moodle/course/view.php?id=34

http://moodlemoot2008.vet-nantes.fr/moodle/course/view.php?id=56

Cheers.

Jerome Mouneyrac - 20/Jun/08 05:13 PM
Hi all,
I would need more information about your work on Web Services for Moodle 2.0.

I went to have a look to the contrib folder in patches/dfws.
Tell me when I'm wrong:
* Functions defined in the Moodle 2.0 specification are in the "apis" repository
* "inout" repository contains all way to access to these functions (SOAP, XML-RPC, REST)
* The repository mod is for module web services (not in the Moodle 2.0 specification yet)
* There is an api.php, connect.php and lib.php file in the project root repository.

It's apparently not the last version of your work, can I get an access to the last version?
Who works on what exactly?
Do we really need all this code for Moodle 2.0? (it could be a stupid question but I read somewhere that it was first based to work with Moodle 1.8)
Does the last code version follow the Moodle 2.0 specification?

For information, I'm still a recent Moodler, and not an expert in web services yet.
I'm a bit confuse, I'd really appreciate more information.

Have you a way to communicate easily? You can contact me on skype, my skype account is jerome.mouneyrac@skype.com

Last point, I can not access to the two French discussions. I created a "jerome" account on the French Moodlemoot website but I'm not allowed to access to these conversations.

Thanks a lot for your help.

Patrick Pollet - 24/Jun/08 09:01 PM
Hello,

   I finally succeeded to get dwfs running against a test Moodle 1.9 ... First the webservice directory MUST be at the "root" of the Moodle installation, nowhere else ...

  The excellent point about this implementation is that the WSDL required by SOAP client in generated "on the fly" by collecting all datatypes and functions declarations from the files present in directories apis/* and mod/* , but they use the outdated nusoap library. A possible issue will be performance since the WSDL will be recomputed at each call.

  Currently I got some answers from :

  - http://mymoodle/webservice/inout/direct/client.php after editing line 4 of this script to give an admin login/password or an user with manual access. The call mdl_course_get_courses_by_user gives no anwser for user id 4, where it should give 2 courses...

   - the python SOAP client navega.py do work, but again I got no answers at get_my_courses.

  - http://mymoodle/webservice/inout/soap/client.php does not run due to a broken nusoap.php file in webservice/inout/soap/nusoap , an hardcoded URL string and a wrong second argument (true)...

 
     It is still unclear to me why there is two entry points (connect.php and inout.php ) and how the security is implemented , i.e how a single call to connect is required before jumping to inout.php and how inout.php reject unconnetd operations... but I am still searching ;-)


     By quickly peeking in apis directory, I noticed that all functions that address creating or deleting Moodle entities (course, user...) do not check for roles... and call directly the delete_course() or delete_user() APi functions.


     As far as the specifications are concerned it is true that the functions do not have yet the proposed names
but that could be easily solved. A more important problem is the parameters ; the calls only use Moodle internal IDS to identify the objects, where it is clear that for external information systems, these values are irrelevant. External system should not keep track of these numbers and talk to Moodle using "idnumbers" at least for users and courses. We have suggested in the specs that all calls should include a parameter stating what field of Moodle should be used to identify something, with a default to "idnumber".
 

   I think Ludo's team should concentrated on writing the code in apis/ and mods/ directory, with the help of Moodle core developers, that are the most competent people on that matter, using the inout/direct/* scripts to test them out and let others take care of the real clients. Currently, people that already have implemented WS for Moodle ( Frederic and me for SOAP and Linux Box Corporation for XML-RPC) and eager to replace their "low level layer" that talks directly to Moodle API or do SQL calls by a set of well established functions located in Ludo's apis/ and mod/ directories

 
My 2 cents...

Cheers


Patrick Pollet - 24/Jun/08 09:11 PM
I forgot to say that there are some interesting docs at http://blogs.dfwikilabs.org/moodle_ws/.
Some entries are in spanish, but Google translator has been my friend as always ;-)

Patrick Pollet - 25/Jun/08 01:52 AM
One more step to do :

Python client navega.py do work as expected if you add the table mdl_api_session to Moodle database.
Copy & paste webservice/webservice/inout/soap/base_de_dades.txt file content into phpmyadmin

CREATE TABLE mdl_api_session (
  id bigint(10) unsigned NOT NULL auto_increment,
  userid bigint(10) unsigned NOT NULL default '0',
  username varchar(100) NOT NULL default '',
  ip varchar(25) NOT NULL default '',
  content text NOT NULL default '',
  session_id varchar(40) NOT NULL default '',
  expires bigint(10) unsigned NOT NULL default '0',
  PRIMARY KEY (id)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='Store session data from usemdl_api_sessionrs migrating to other sites' AUTO_INCREMENT=1

So I start to understand how security is implemented. Operation mdl_config_login located in connect.php add a record to mdl_api_session table and that record is fetched back by inout.php and deleted by a call to mdl_config_logout operation.

In a prod version, database updates should be done by standard Moodle manipulations .

A+

Jerome Mouneyrac - 25/Jun/08 02:34 PM
Thanks a lot for all these information Patrick :)

Patrick Pollet - 26/Jun/08 05:58 PM
Hello,

   I played harder on current dfws implementation and start to have some good results with it ...

Attached a modified version of cvs:/moodle.org/contrib/patches/dfws/webservice.
Unpack it at the root of you Mooalde and read the README_IMPORTANT.TXT for security.
--------------------------------------------------------------------

- My goal was ONLY to try to understand how it works and check that adding operations
was easier than in other implementations. That is TRUE !

- a basic api for categories has been added in apis/ to test adding operations.
Currently there is only one entry : mdl_category_get_categories(parent,recurse). DONE

- UTF-8 encoding of the WSDL is enforced if Moodle's database is unicode. DONE

- the current "security mechanism" by requiring first a call to mdl_config_login(user,pwd)
and then passing back the sessionid to all calls to inout.php has been removed since it does not work.
Anymore can call an inout operation that do not return a "complex type" such as mdl_user_add_instance
without authentfication. DONE

 I would suggest that the sessionid be added as the first parameter to all calls
but mdl_config_login and tested against the database... like it is currently done in OK Tech WS implementation. TODO

-since there is currently TWO entry points (connect.php and inout.php) the service names MUST be different
to allow automatic classes generation by wsdl2xxx utilities without overwriting ...DONE

- for testing SOAP clients, see the directory inout/soap/testing and the file README_IMPORTANT.TXT.



Patrick Pollet - 26/Jun/08 05:59 PM
Some modificatiions to CVS current release (date APril 18th).

Patrick Pollet - 26/Jun/08 10:57 PM
correct some typos in first version attached.

Can an user with admin privileges remove the previous version. Thkx

Ludo ( Marc Alier) - 28/Jun/08 11:13 PM
In this subtask http://tracker.moodle.org/browse/MDL-13127 I and David are talking about the parameters for the delete_user function BUT the issue here migth affect all the API because
we should have a consistent way of passing parameters. Shall we debate it there?

Jerome Mouneyrac - 30/Jun/08 10:42 AM
I don't get it. Can you post an example when passing parameters is not consistent?
The technical specification (http://docs.moodle.org/en/Development:Web_services) should mention the parameters for every functions. Let us know where the specs are incomplete.

Ludo ( Marc Alier) - 30/Jun/08 02:10 PM
Sorry,
I did not explain myself - language issues :-( - What I mean is that the way parameters are passed should be alike in all functions.

Martin Dougiamas - 30/Jun/08 02:34 PM - edited
Absolutely they should be consistent.

I like the idea of passing an array of "field" -> value like:

array (
   'idnumber' => '434343',
   'email' => 'martin@fred.com',
   'id' => '3'
)

Where they are processed in order. Does that make sense to all?

  delete user with idnumber 434343
  delete user with email martin@fred.com
  delete user with id 3

Obviously this could delete a lot of users at once:

array (
   'firstname' => 'Martin'
}

Perhaps we should allow wildcards too:

array (
   'lastname' => 'Doug%'
}

Ludo ( Marc Alier) - 30/Jun/08 02:44 PM
Hmm... acording to this who do we delete 3 users with id 4,5,6 ... the position of the associative array "id" s is taken...
So how do we doi it? separating id's by commas?

Ludo ( Marc Alier) - 30/Jun/08 02:49 PM
My idea is that we should implement ONE function to do all the heavylift ( delet the user and the role asocciations and so on ) ... this function shuld be invoqued by 4 functions like delete_users_by_id delete_users_by_username ...
So it may be easy for the user developer and from the maintenance point because just one function

Patrick Pollet - 30/Jun/08 02:59 PM
>In any order, and any number or items, where they are logically joined with AND in the function. Does that make sense to all?

  So if you mean delete user whose idnumber is 434343 AND email=fred@xxx AND id=3 ??? :In that case is it too much demanding on the external SIS that will likely not know anything about Moodle internal ids and does not want to know about them.

  And if you mean user whose idnumber is 434343 OR email=fred@xxx OR id=3 , it does not make sense to me either since, when writing processes to sync Moodle with my SIS it is very unlikely that I shall scan my database twice, once for idnumber and one for email. If in my place idNumber are the "primary key", I shall use it anywhere, if it is the email, I shall uses is instead, but never both...

 I think Ludo's question is more :

    -Shall we have ONE public operation delete_users (array of some info, corresponding field name in Moodle ) that could be called as :
     res = delete_users (array(1,2,3), 'id')
or res = delete_users (array('CS27001','CS27002',...), 'idnumber')
 or res = delete_users (array(jack@here,fred@ther,...), ,'email')
or res = delete_users (array(jack,fred,...), 'username')

  - Or shall we have four public operations :

        res = delete_users_byid (array(1,2,3')
or res = delete_users_byidnumber('CS27001','CS27002',...)
 or res = delete_users_byemail (array(jack@here,fred@ther,...)
or res = delete_users_byusername (array(jack,fred,...)

 and make delete_users private to avoid devastating calls such as delete_users (array(1),'confirmed') or delete_users(array('patrick'),'firstname')... unless you feel that it should be allowed

  Also the question of wildcards should be addressed (permitted or not) such as delete_users (array('CS%','idnumber) ....


   To me it should not be permitted to delete_users for safety reasons ; it that case we should request a true identifying id (unique in database) ;
in contrast it should be allowed in get_users . I shall be happy with a get_users(array(1),'deleted')
    

Cheers.