Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

Tip: Batch calculations with Matlab + Comsol 4.0a

Please login with a confirmed email address before reporting spam

Hi,

I just wanted to share my experience regarding batch-type (i.e., non-interactive) calculations using comsol 4.0a and the matlab livelink. I lost quite some time trying to run a comsol matlab script (.m file) on a linux cluster, and I thought some people might be interested in hearing the workaround I found.

When working with the cluster, I have two major constraints:
1. I do not have any kind of display (X) support,
2. I have to submit the calculation via a simple bash script.


In 3.5a, running a .m file in batch mode was very easy (At the very least, it was nicely documented : www.comsol.com/support/knowledgebase/980/), One would use:

comsol matlab -ml -nodesktop -ml -nosplash -mlr "myscript, exit"

In comsol 4.0a, using something along the lines of

comsol server matlab -nodesktop -nosplash -r myfile

does not lead to similar behavior. First, an xterm is spawned, and the 'nodesktop' matlab runs in this new terminal. The '-r myfile' option is ignored, and matlab does not run my .m file on startup.

To work around the fact that 'comsol server matlab' spawns an xterm, I was told by comsol support to do the following:

1. Start the comsol server
2. Start matlab, and connect to the server using mphstart



To be able to do this within one bash script, I basically had to launch the server in the background. However, I could not use

comsol server &
matlab -nodesktop -nosplash -r myfile

as the server process would stop before matlab could connect to it whenever it was launched as a background process. By using instead

comsol server </dev/null > /dev/null &
matlab -nodesktop -nosplash -r myfile

instead of simply "comsol server &" then the server process is not stopped and you can run matlab, connect to comsol server, and run your .m script, without any user-intervention nor need for X support. Also, the comsol server exits when running ModelUtil.disconnect in Matlab, so I don't leave any process on the compute node..

Hoping this might be useful for someone ;)
By the way, if anyone has a better (cleaner?) way of achieving the same thing, I'd be happy to hear it!

Charles




24 Replies Last Post 28 apr. 2017 05:52 EDT
Ivar Kjelberg CSEM SA (CH)

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 10 sep. 2010 05:06 EDT
Hi

thanks for the tips, I start too to have large models I would like to run batch over the night/week-end so this will be usefull.

Hope COMSOL will make it cleaner/easier in next release

--
Good luck
Ivar
Hi thanks for the tips, I start too to have large models I would like to run batch over the night/week-end so this will be usefull. Hope COMSOL will make it cleaner/easier in next release -- Good luck Ivar

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 14 sep. 2010 06:18 EDT
This is a very good tip, i just need it to solve my problem, but as i tried as you suggested.

the command
$comsol server < /dev/null > /dev/null &
does work, while after running this command, you can see there is a 'comsollauncher' process
running in the background, also consuming quite a lot of cpu, though nothing is running at all.
then i run the command
$matlab -nodesktop -nosplash -r file
it gives me an error like
'??? Undefined function or variable 'ModelUtil'.'

Do you guys know what's wrong with it?

Thanks in advance.

This is a very good tip, i just need it to solve my problem, but as i tried as you suggested. the command $comsol server < /dev/null > /dev/null & does work, while after running this command, you can see there is a 'comsollauncher' process running in the background, also consuming quite a lot of cpu, though nothing is running at all. then i run the command $matlab -nodesktop -nosplash -r file it gives me an error like '??? Undefined function or variable 'ModelUtil'.' Do you guys know what's wrong with it? Thanks in advance.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 16 sep. 2010 14:01 EDT
Make sure that you have your COMSOL40a\mli directory added to the Matlab path.
Make sure that you have your COMSOL40a\mli directory added to the Matlab path.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 6 dec. 2010 21:14 EST
This is a great tip!

Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link.

Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in:

#!/bin/bash
comsol server </dev/null> /dev/null &
matlab -nodesktop -nosplash -r 'mphstart'

If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname).

Finally, you need to give everyone access to run this program, so back in the /usr/bin directory,

chmod +x /usr/bin/shortcutname
This is a great tip! Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link. Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in: #!/bin/bash comsol server /dev/null & matlab -nodesktop -nosplash -r 'mphstart' If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname). Finally, you need to give everyone access to run this program, so back in the /usr/bin directory, chmod +x /usr/bin/shortcutname

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 18 jan. 2011 11:43 EST
I won't say it is a cleaner way, but it is more 'transparent' so to say for many people to start comsol's server like this:

comsol server > server.txt &

This way whatever output comsol server wants to spew will be dumped in server.txt file. And honestly, sometime during debugging it is useful to know what it has to say :)
I won't say it is a cleaner way, but it is more 'transparent' so to say for many people to start comsol's server like this: comsol server > server.txt & This way whatever output comsol server wants to spew will be dumped in server.txt file. And honestly, sometime during debugging it is useful to know what it has to say :)

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 6 apr. 2011 08:39 EDT
Good tip, that also works for Comsol 4.1

But nevertheless: very inconvenient to be honest..
Hopefully there is a better solution in the next version.
Good tip, that also works for Comsol 4.1 But nevertheless: very inconvenient to be honest.. Hopefully there is a better solution in the next version.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 4 mei 2011 00:22 EDT
Thank you very much, Charles!

I've been trying to run Comsol 4.0a in batch mode on our cluster for like a year (periodically dropping it and coming back to it again and involving our cluster and comsol tech support with no results). I've just tested what you suggested - your method works nicely! Finally!!!

Sincerely,
Alex.
Thank you very much, Charles! I've been trying to run Comsol 4.0a in batch mode on our cluster for like a year (periodically dropping it and coming back to it again and involving our cluster and comsol tech support with no results). I've just tested what you suggested - your method works nicely! Finally!!! Sincerely, Alex.

Muhammad Mohsin Rehman

Please login with a confirmed email address before reporting spam

Posted: 10 years ago 25 okt. 2011 23:29 EDT
Updated: 3 years ago 20 mrt. 2018 10:49 EDT
Hi Charles,

Do you have any idea of running batch process on windows?
I have two files and I want to run them one by one in the way that results of one should be used as initial conditions for the other, is there any way of doing it?

Thanks in advance for any of your help!
Hi Charles, Do you have any idea of running batch process on windows? I have two files and I want to run them one by one in the way that results of one should be used as initial conditions for the other, is there any way of doing it? Thanks in advance for any of your help!

Please login with a confirmed email address before reporting spam

Posted: 10 years ago 26 okt. 2011 08:05 EDT
Hi Muhammad,

Sorry, I cannot really help you with running batch calculations under Windows. I mainly use my employer-issued Windows laptop as a glorified email client, and therefore don't have much experience with Comsol under Windows.

Regarding the second part of your question; could you not simply merge the two files into one, using the second file as a second study step? In that case it would be straightforward to setup the calculation in such a way that the results of the first calculation can be used as initial conditions for the second calculation..

Good luck,
Charles
Hi Muhammad, Sorry, I cannot really help you with running batch calculations under Windows. I mainly use my employer-issued Windows laptop as a glorified email client, and therefore don't have much experience with Comsol under Windows. Regarding the second part of your question; could you not simply merge the two files into one, using the second file as a second study step? In that case it would be straightforward to setup the calculation in such a way that the results of the first calculation can be used as initial conditions for the second calculation.. Good luck, Charles

Muhammad Mohsin Rehman

Please login with a confirmed email address before reporting spam

Posted: 10 years ago 26 okt. 2011 09:19 EDT
Thanks a lot Charles for your feedback!

I have defined the second study step in the same file but could not find any option in the second study to set the results of first study as initial values.(Please see the attached image for the options)

Do you mean that I need to define the whole model again?

Can you please elaborate what do you mean by setting the second file as second study step?

Thanks in advance!
Thanks a lot Charles for your feedback! I have defined the second study step in the same file but could not find any option in the second study to set the results of first study as initial values.(Please see the attached image for the options) Do you mean that I need to define the whole model again? Can you please elaborate what do you mean by setting the second file as second study step? Thanks in advance!


Please login with a confirmed email address before reporting spam

Posted: 9 years ago 11 mei 2012 15:56 EDT
The above worked for me on 4.2a. Useful tips, many thanks. Saved me a wasting an afternoon.
The above worked for me on 4.2a. Useful tips, many thanks. Saved me a wasting an afternoon.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 17 apr. 2014 05:31 EDT
Hi, I hope this thread is still active.
I am having big troubles in implementing your suggestions. I prepared a run_script.sh file with:

#!/bin/sh
comsol server </dev/null > /dev/null &
bsub matlab -nodesktop -nosplash -r "addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; study1"

wher study 1 is the fiel that I actually need to run. Then I submit the job on the cluster I have access to with

bsub < run_script.sh

but I get an error as if I didn't start the comsol server:

Error using mphstart (line 152)
A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart

How do I fix this?


This is a great tip!

Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link.

Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in:

#!/bin/bash
comsol server </dev/null> /dev/null &
matlab -nodesktop -nosplash -r 'mphstart'

If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname).

Finally, you need to give everyone access to run this program, so back in the /usr/bin directory,

chmod +x /usr/bin/shortcutname


Hi, I hope this thread is still active. I am having big troubles in implementing your suggestions. I prepared a run_script.sh file with: #!/bin/sh comsol server /dev/null & bsub matlab -nodesktop -nosplash -r "addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; study1" wher study 1 is the fiel that I actually need to run. Then I submit the job on the cluster I have access to with bsub < run_script.sh but I get an error as if I didn't start the comsol server: Error using mphstart (line 152) A connection to Comsol could not be established. Please check that a Comsol Server is started prior to calling mphstart How do I fix this? [QUOTE] This is a great tip! Perhaps one step better, for people running OSX - I've saved the whole thing into it's own symbolic link. Navigate to /usr/bin, create a file with the text editor of your choice (I used nano, for simplicity's sake). In the file, put in: #!/bin/bash comsol server /dev/null & matlab -nodesktop -nosplash -r 'mphstart' If it gives you trouble, you might have to use a sudo command to get it to create (as in sudo nano shortcutname). Finally, you need to give everyone access to run this program, so back in the /usr/bin directory, chmod +x /usr/bin/shortcutname [/QUOTE]

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 17 apr. 2014 05:43 EDT
Hi Stefano

Just to be sure, did you start the comsol server at least once on the node? The first time you start it, you need to (interactively ) setup a username/password. As far as I know, this cannot be done directly using a script. Luckily, you only need to do it once..

Charles

Hi Stefano Just to be sure, did you start the comsol server at least once on the node? The first time you start it, you need to (interactively ) setup a username/password. As far as I know, this cannot be done directly using a script. Luckily, you only need to do it once.. Charles

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 17 apr. 2014 06:20 EDT
actually, problem solved. There was a mistake in my file.
Thanks for the answer anyway.
actually, problem solved. There was a mistake in my file. Thanks for the answer anyway.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 14 jul. 2014 07:48 EDT
Hi Stefano,
does your script also work when it is started multiple times? I'm wondering if this is possible without setting a distinct port in order to connect an instance of MATLAB with a certain Comsol server.
Hi Stefano, does your script also work when it is started multiple times? I'm wondering if this is possible without setting a distinct port in order to connect an instance of MATLAB with a certain Comsol server.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 14 jul. 2014 11:34 EDT
Hi Eli,
I haven't being launching simulations from matlab in a while, so I had to go back to old scripts :)

I simply launched the following script

#!/bin/sh
comsol server </dev/null > /dev/null &
env > $HOME/env.log
matlab -nodesktop -nosplash -r "disp('start'); addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; disp('done')" > mphstart.log

So, it is just opening a connection with the server and just displaying something.
I launched 2 of them and it seems to work.
Hi Eli, I haven't being launching simulations from matlab in a while, so I had to go back to old scripts :) I simply launched the following script #!/bin/sh comsol server /dev/null & env > $HOME/env.log matlab -nodesktop -nosplash -r "disp('start'); addpath(genpath('/cluster/apps/comsol/4.4/mli')); mphstart; disp('done')" > mphstart.log So, it is just opening a connection with the server and just displaying something. I launched 2 of them and it seems to work.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 15 jul. 2014 11:12 EDT
Hi,
thank you fopr providing this. I use something similar, but because I want to run many models in parallel, there are lots of MATLAB instances and Comsol servers running at the same time. In order to make MATLAB connect to a distinct Comsol server I need to provide a port number when starting up the server:

comsol server -port 2037 </dev/null > /dev/null &

And then connect MATLAB to this Comsol server:

matlab -nodesktop -nosplash -r " addpath(genpath('path_to_mli')); mphstart(2037); someFunction;exit;" > matlab_output.txt

I guess without the port number MATLAB would connect to some random Comsol server.
Hi, thank you fopr providing this. I use something similar, but because I want to run many models in parallel, there are lots of MATLAB instances and Comsol servers running at the same time. In order to make MATLAB connect to a distinct Comsol server I need to provide a port number when starting up the server: comsol server -port 2037 /dev/null & And then connect MATLAB to this Comsol server: matlab -nodesktop -nosplash -r " addpath(genpath('path_to_mli')); mphstart(2037); someFunction;exit;" > matlab_output.txt I guess without the port number MATLAB would connect to some random Comsol server.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 16 jul. 2014 07:59 EDT
If you don't specify the port number Comsol will create a server with the default port number 2036. If that's already busy It will use 2037 and so on. I am not specifying the port number and that seems to work even in batch mode.

Your method is also fine, I guess you need to specify a different port number for each job? But I think that's not difficult to implement.
If you don't specify the port number Comsol will create a server with the default port number 2036. If that's already busy It will use 2037 and so on. I am not specifying the port number and that seems to work even in batch mode. Your method is also fine, I guess you need to specify a different port number for each job? But I think that's not difficult to implement.

Please login with a confirmed email address before reporting spam

Posted: 7 years ago 18 jul. 2014 16:53 EDT
If I open several instances of MATLAB and Comsol server in a sequence without giving MATLAB a distinct port to connect to, it might happen that e.g. two MATLABs connect to the same Comsol server. This would produce errors as soon as both want to run models on that server at the same time. Of course if there are only two MATLAB-Comsol pairs opened this double connect to one server is not liekly to happen.
If I open several instances of MATLAB and Comsol server in a sequence without giving MATLAB a distinct port to connect to, it might happen that e.g. two MATLABs connect to the same Comsol server. This would produce errors as soon as both want to run models on that server at the same time. Of course if there are only two MATLAB-Comsol pairs opened this double connect to one server is not liekly to happen.

Please login with a confirmed email address before reporting spam

Posted: 6 years ago 2 apr. 2015 13:58 EDT
Hi,

I am not sure whether the post is still active. I tried the following in out linux server:

#!/bin/sh
comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt

These lines are saved in a file named run1.s and then I run the file using the command

nohup sh ./run1.s &

I receive the error: Suspended (tty output) sh ./run1.s

I tried using the following two lines in command line without using a shell script and it works fine:

comsol server </dev/null >/dev/null &
matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt

Can anyone help?
Hi, I am not sure whether the post is still active. I tried the following in out linux server: #!/bin/sh comsol server /dev/null & matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt These lines are saved in a file named run1.s and then I run the file using the command nohup sh ./run1.s & I receive the error: Suspended (tty output) sh ./run1.s I tried using the following two lines in command line without using a shell script and it works fine: comsol server /dev/null & matlab -nodesktop -nosplash -r "disp('start');addpath(genpath('/apps/comsol4.4/mli'));mphstart;disp('done');exit;" > OUT.txt Can anyone help?

Please login with a confirmed email address before reporting spam

Posted: 5 years ago 18 nov. 2015 10:44 EST
Things not working for me

comsol server </dev/null > /dev/null &

I was expecting to see some messages showing the server is starting up. I waited about 3min but nothing showed up. Then I proceeded adding a wait time
$sleep 120

and the run

matlab -nodesktop -nosplash -r myfile


After matlab starts, I ran
>> addpath((genpath(<path/to/comsol44/mli>))
>>mphstart

And matlab complained with
"A connection to Comsol could not be established. Please check that a Comsol
Server is started prior to calling mphstart"

Something might have gone wrong during the server startup but how to I correct it?

I wonder what exactly this means

</dev/null > /dev/null


should I be replacing </dev/null> with some other path? I am confused
Things not working for me [QUOTE] comsol server /dev/null & [/QUOTE] I was expecting to see some messages showing the server is starting up. I waited about 3min but nothing showed up. Then I proceeded adding a wait time $sleep 120 and the run [QUOTE] matlab -nodesktop -nosplash -r myfile [/QUOTE] After matlab starts, I ran >> addpath((genpath()) >>mphstart And matlab complained with "A connection to Comsol could not be established. Please check that a Comsol Server is started prior to calling mphstart" Something might have gone wrong during the server startup but how to I correct it? I wonder what exactly this means [QUOTE] /dev/null [/QUOTE] should I be replacing with some other path? I am confused

Please login with a confirmed email address before reporting spam

Posted: 5 years ago 22 nov. 2015 13:45 EST
Hi Camille,
'> /dev/null &' sends all output of the starting Comsol server to /dev/null which means it will not be visible to you, the process is then send to background (via '&') so that the terminal is free to proceed to the next command for starting MATLAB. A better way is it to send the output to a log-file using 'Comsol server > /dev/null 2>&1 > serverLog.txt &'.
Alternatively you may also just start Comsol server in one terminal window and then open another one and start MATLAB from there. This also enables you to monitor the output of Comsol server. It should show you when a MATLAB instance connects.
Best,
Eli
Hi Camille, '> /dev/null &' sends all output of the starting Comsol server to /dev/null which means it will not be visible to you, the process is then send to background (via '&') so that the terminal is free to proceed to the next command for starting MATLAB. A better way is it to send the output to a log-file using 'Comsol server > /dev/null 2>&1 > serverLog.txt &'. Alternatively you may also just start Comsol server in one terminal window and then open another one and start MATLAB from there. This also enables you to monitor the output of Comsol server. It should show you when a MATLAB instance connects. Best, Eli

Please login with a confirmed email address before reporting spam

Posted: 5 years ago 11 nov. 2016 10:40 EST
Thanks Charles Doiron! Works on 5.2a
Thanks Charles Doiron! Works on 5.2a

Please login with a confirmed email address before reporting spam

Posted: 4 years ago 28 apr. 2017 05:52 EDT
This post was extremly useful, thank you very much. However I'd like to add something, that will be probably mostly useful for students like me, who don't have superuser rights (and might not be big linux gurus either.)

A lot of times if Comsol Server is started with e.g. "comsol-52a server &", it just exits, and no chance to connect to it with MATLAB. (If the log is redirected to a txt, one can see that it waits for username and password, and it exits, as none is provided.) So what I suggest is opening up a screen, start a Comsol Server there, with e.g. "comsol-52a server", and look at the output: a lot of times the port it connects to is not 2036, but 2037, 2038, etc. You're free to detach from the screen now.

Now, open up a new screen, start MATLAB, without X forwarding enabled, it then displays its command promt. (And does not exit with an error, fortunately.) Add the "mli" library to the MATLAB path, like "addpath(''/usr/local/comsol52a/multiphysics/mli')" and now run "mphstart", but also specify the port number used: "mphstart(2037)" (if localhost:2037 was the port number displayed by Comsol Server previously.) Without passing the port number as an argument, it will just throw an error, because it always tries to connect to port 2036 by default. Now you can start any Comsol script and detach from the screen, it will keep running.

In case of other problems, it is worth opening the "mphstart.m" file in the "mli" library previously added, it lists a lot of other arguments "mphstart" is able to receive.

In case somebody can suggest a nicer solution, e.g. without using two screens, please share.
This post was extremly useful, thank you very much. However I'd like to add something, that will be probably mostly useful for students like me, who don't have superuser rights (and might not be big linux gurus either.) A lot of times if Comsol Server is started with e.g. "comsol-52a server &", it just exits, and no chance to connect to it with MATLAB. (If the log is redirected to a txt, one can see that it waits for username and password, and it exits, as none is provided.) So what I suggest is opening up a screen, start a Comsol Server there, with e.g. "comsol-52a server", and look at the output: a lot of times the port it connects to is not 2036, but 2037, 2038, etc. You're free to detach from the screen now. Now, open up a new screen, start MATLAB, without X forwarding enabled, it then displays its command promt. (And does not exit with an error, fortunately.) Add the "mli" library to the MATLAB path, like "addpath(''/usr/local/comsol52a/multiphysics/mli')" and now run "mphstart", but also specify the port number used: "mphstart(2037)" (if localhost:2037 was the port number displayed by Comsol Server previously.) Without passing the port number as an argument, it will just throw an error, because it always tries to connect to port 2036 by default. Now you can start any Comsol script and detach from the screen, it will keep running. In case of other problems, it is worth opening the "mphstart.m" file in the "mli" library previously added, it lists a lot of other arguments "mphstart" is able to receive. In case somebody can suggest a nicer solution, e.g. without using two screens, please share.

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.