Cancel
Showing results for 
Search instead for 
Did you mean: 

Constant/same results from ExperimentManager

Experimenter
Experimenter

Good evening,

 

I am currently doing some experiments with my model via the ExperimentManager.

If I repeat a run with my model via the EventController, I get unique results every time (keeping all parameters the same). However if I use the ExperimentManager to repeat an exact same run, I always get the same results (again keeping all parameter the same).

 

In the EventController I applied the 'Increment Variant on Reset' option, which seems to work for seperate runs by hand creating unique results every time. This option doesn't seem to work for the ExperimentManager.

 

For example, if one observation is applied for an experiment I get the result '3'. If two observations are applied the result will become '3' and '5'. For three observations the results are '3', '5' and '4'. I would like to get unique results every observation doing the same experiment, for example first the result '4', then '5' and '2' and with three observations '3', '4' and '1'.

 

It might have something to do with the settings of the source, I have attached the applied distribution which should give a creation interval according to a distribution. On the other hand it does work if I run the model by hand.

 

Thanks in advance for the support.

6 REPLIES 6

Re: Constant/same results from ExperimentManager

Siemens Phenom Siemens Phenom
Siemens Phenom

How many observations per run do you use when running the experiment manager? Because different observations of the same experiment should yield different result in each observation, just as when you do it manually. Like this:

ExpManager.png

Re: Constant/same results from ExperimentManager

Pioneer
Pioneer

@MarcusA, thank you for your reply.

I once had the same problem that @PiJoGi40 had and I could not figure it out as well.

You are right, if the number of observations is increased, random values are ouputted which is good. However, if you reset the experiment manager and run the same experiment with the same number of observations, the same random results are obtained again. 

For example, for your attached image, if you would run the experiment manager again, you will get the same results for the ten outputs time after time. And if for example, you would decrease your observations to three and run the experiment manager again, will the output be 3752, 3752 and 3716? (Which is the first three results of the ten you showed in the attached image)

What I think @PiJoGi40 wants is that every experiment manager run, for the same experiment with and the same number of observations, the results vary every run (like it does when you run the model by hand using eventcontroller). I think that is what he also means with the increment variant on reset doesnt work for experiment manager.

Is this because the dice has to be rolled for the source formula for example? I eventually solved it to do multiple experiments with the same input values which I think should not be the way for this to work.

 

Kind regards,

 

Niek

 

Re: Constant/same results from ExperimentManager

Siemens Phenom Siemens Phenom
Siemens Phenom

@PiJoGi40, @Niek: I decided too look at the documentation for the experiment manager, and in the chapter Setting the Random Number Seed Values I found this: 

"The ExperimentManager only works correctly when Increment Variant on Reset in the EventController in your simulation model is deactivated"

 

So if you deactivate "Increment Variant on Reset" in your EventController, the Experiment manager should work as intended. Don't ask me why, because i don't have a clue Cat Wink

Re: Constant/same results from ExperimentManager

Legend
Legend

THis made me curious and after digging deeper I found the UserSeedValueMethod...

 

 

Capture1.PNG

This should solve the problem of having the same results for different experiments.

Apparently, during the normal operation of the Experiment Manager the seed value in Event Controller is incremented only once.

 

Using this method according to the UserSeedValueMethodTemplate seemlingly changes that for every observation.  

 

Edit: I forgot to attach an image of it.. Here it is

 

Capture.PNG

 

Re: Constant/same results from ExperimentManager

Pioneer
Pioneer

Thank you @MarcusA and @Sanchay for digging deeper into this.

I can't verify if this works for my model that had the same problem since this was for a small project last year. @PiJoGi40 , could you verify if this solution works?

 

Niek

Re: Constant/same results from ExperimentManager

Experimenter
Experimenter

Good afternoon @Sanchay@MarcusA and @Niek 

 

Thank you all for the support.

 

I first tried the suggestion of @MarcusA, by deactivating the Increment Variant on Reset setting in the EventController. Unfortunetaly this didn't work.

 

I then tried the UserSeedValueMethodTemplate suggested by @Sanchay, but the standard template didn't work either. I started looking into more detail into the random numbers variant reset and found something that does work in the UserSeedValueMethod:

eventController.IncrementRandomNumbersVariantOnReset := true

This seems to enable the IncrementRandomNumberVariant on every (new) run of the ExperimentManager.

 

By implementing this code in the method of the ExperimentManager I always get a unique result for exactly the same run. Exactly what I was looking for.

I have attached a screenshot of the implemented code in the UserSeedValueMethod.

 

Thank you all for helping me out!