Follow the steps to reproduce and make sure you reach the expected behaviour. Manual Steps (URL) Log into site as admin Go to "site home" (in navigation block) Turn editing on (Administration block > front page settings) In the Main Menu block "add an activity or resource" and select URL Fill out the form and make sure the "Display" (under Appearance category) is set to "In pop-up" Save and return to course Turn editing off After the URL resource was added to the Main Menu block, confirm that the javascript executed on the page correctly You can look at the developer console in the browser to make sure it hasn't errored You can also confirm that the "add an activity or resource" link is actually created in the Main Menu block, since it is javascript that converts the text to a hyperlink. Verify that the activity link works as expected ("In pop-up"). Turn editing on Edit the activity and change "Display" to "In new window". Save and return to course. Without turning editing off , verify that the activity link works as expected ("In new window"). Manual Steps (File) Repeat the steps above with a file resource, swapping the "In pop-up" and "In new window", so both get tested with editing on and off. Automatic Run the behat test blocks/site_main_menu/tests/behat/add_url.feature