Community Bonding Period - Second Meeting and Weekly work update

-

Hi,


Last week on the 19th of May, I met with the mentors again for the second time on Zoom. However, one of my primary mentors, Andreas Dräger, was meeting for the first time as he was unable to attend the first meeting due to the Ph.D. defense. It was great meeting him after a long discussion on the mails. Now, here's the summary of the meeting with the weekly work done.

  • Mentors confirmed in the meeting that the comp tests' issues were due to the CompFlatteningConverter class of JSBML. So, we decided to skip the tests currently and move forward until the issue gets solved in the JSBML repository. Matthias made an issue for the same in the SBSCL repository (issue #36) as well as the JSBML repository (JSBML issue #213).

  • And, PR #37 was merged in the SBSCL repository with skipping the comp tests. Also, build with oracle JDK 11 was added to Travis CI.

  • Then, we discussed that I should start working on the first milestone for the project, i.e., upgrading SBSCL to support SBML Level 3 Version 2.

  • To start with this, we needed first to run the SBMLTestSuite tests successfully with the current SBSCL version. But, we found that the SBMLTestSuite tests were skipped currently in the build. So, this was the main task for the week to work on. Matthias created an issue #35 for this too.

  • I started to work on the SBMLTestSuite tests. Then, I resolved some small issues, and the tests started to run, but many of them failed.

  • Then, we found out most of the tests were failing due to comparing the results with the predefined results.

  • Some of the cases failed as they were comp tests and were simulated with SBMLSimulator. I updated them to run with CompSimulator, but some still failed due to the existing error from the JSBML class.

  • Issues were created for a single test-case, failed due to Arithmetic Exception (issue #38) while for one other, which took a long time to simulate (currently, skipped) (issue #39).

  • Currently, the results are compared with the predefined results with the RelativeEuclideanDistance, which mentors told is not a useful comparison metric. And, even the threshold was given a constant (0.2), which was not accurate.

  • So, we decided to change the metric to the maximum absolute distance for each of the variables in the model. I added a class MaxAbsDistance for this. But, we had errors in this too.

  • After this, the mentors guided me that the combination of the max absolute distance and max relative distance is the best way for comparisons as this is how integrators work. After this discussion, I added the MaxRelDistance and changed the thresholds as variables as per the absolute (for MaxAbsDistance) and relative (for MaxRelDistance) tolerances for the reactions. However, the results even had issues with these thresholds. Then I had my 3rd meeting with mentors, which I will be elaborating on in my next blog with further updates on SBMLTestSuite tests!


The work for the SBMLTestSuite tests is done in the testrunner branch in the forked repository (Link given below).
The forked repository link (where the updated code stays): https://github.com/hemilpanchiwala/SBSCL


Regards!
Location: Ahmedabad, Gujarat, India

Comments

Post a Comment

Popular posts from this blog

Starting Google Summer of Code 2020

-
Image
Hi, I am Hemil Panchiwala, 2nd year CSE student at the Indian Institute of Technology, Roorkee, India. I am incredibly excited to share that I was selected in the Google Summer of Code (GSoC) this year with the National Resource for Network Biology (NRNB) as a mentoring organization. I will be working on the project " Simulation of system biology models in Java: support for new features " in this program. I am incredibly thankful to the wonderful mentors, namely Dr. Andreas Dräger , Dr. Matthias König , Mykola Zakharchuk, and Shalin Shah , who helped me to get selected and will guide me throughout this project. This blog series is dedicated mainly to share my progress for the project. The official coding period starts from 1st June, and the community bonding period has currently begun. In this period, I will be exploring the codebase with solving some small issues and will discuss the implementation details with the mentors and finalize them. What is NRNB? The aim of the
Read more

GSoC Week 10

-
Image
This week I mainly worked on adding the support of the stochastic simulation to SBSCL. And, it is excellent that the simulation is giving great results with most tests passing. This addition of stochastic simulation has improved SBSCL supporting most of the simulation types now, which makes it a fully-featured simulation library. Now, here’s the summary of the work done this week with the topics discussed in the meeting: Mentors suggested writing the work done overview of the week before some hours of the meeting so that they can go through and provide their views in the meeting properly. We then mainly discussed the stochastic simulation implementation (using FERN) and the runtime due to simulating many times to get the mean and standard deviation. We decided to test all the models from the stochastic test suite and update the result format according to the test suite results. I worked on this mainly, and now the output format of the result is updated. The new test file StochasticTest
Read more

GSoC Week 11

-
Last week, I mainly worked on passing the current failing stochastic tests and ran Travis CI successfully. With this, we also started working on refactoring the SBMLinterpreter class, which has become quite a big class doing many things. Here is the quick summary of the updates from the weekly meeting with the work updates: In the meeting, we mainly discussed the failing stochastic tests. We decided to work on them and check whether the results were converging towards the pre-defined results on increasing the number of simulations or not. If they don't, then it was for sure that there is some issue in the implementation. So, I started working on that, and the small issue I found was that while comparing with the pre-defined results, the wrong column was considered from the pre-defined results, which made some tests fail. It is fixed now through this commit . After this, on further checking, we saw that some of the tests were like failing all the time regardless of the number of sim
Read more