How Miria distributes Jobs to Datamovers
Articles / Posts
- Tina 4.8 GA is available
- Lina 6.1 GA is available
- PowerShell for Lina 6.0
- Advisory ID : LINA/ADE-2023-0002
- Advisory ID : LINA/ADE-2023-0001
- Time Navigator 4.6.9 Hyper-V Software Alert
- Tina 4.7.1 GA is available !
- Miria install error on a Linux server without X11
- HSS Replication requires attention when upgrading to 5.x
- LINA Agents 5.3.4
Categories
- Lina – EN (21)
- Miria – EN (8)
- Tina – EN (18)
Archives
- July 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- October 2022
- September 2022
- July 2022
- December 2021
- November 2021
- October 2021
- July 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
Typical Miria configuration uses 1 database server and 1 or more datamovers (or agents).
Pools of 1 or more datamovers can be created
Excluding database server from pools avoids the risk of overloading it as the database server is also an agent (datamover).
The database server records all required information to keep track of migrated files but also orchestrates the migration task(s) and their related job(s)
A migration task transfers files from source storage to destination storage (many types supported by Miria)
Per default, 1 migration task is using 1 job that is using 1 thread.
But you can configure the task to run more than 1 job.
And/or you can configure jobs to run more than 1 thread.
Job load balancing
Starting the migration task will start scanning the source and the first job starts transferring data soon after.
Then, if desired, more jobs can be fired based on criteria below that can be all defined:
As soon one of the criteria is reached, a new job is fired and Miria DB server assigns that new job to the datamover in the pool having the least number of jobs already running.
Miria load job balancing criteria is based on the number of job(s) each datamover is running.
Single thread or Multithread
Per default, 1 job is single thread consuming 1 CPU core and some memory.
But 1 job can also be multithread in what case each thread uses 1 CPU core and some memory.
The number of threads a job can run is a task’s configuration parameter.
Based on that explanation, you can partially understand how a Miria configuration is sized.
Take an example:
Trying to run more jobs or more threads per job will cause performance problem!
If you need more jobs/threads to complete the migration in the timeframe foreseen, you can add more datamover(s) so more jobs/threads can be spread across all of them.
Performance will also be impacted by many other factors like the source and the destination storage’s load capabilities and the network path performance for instance.
The first migration’s run last the longest because :
The next run of the migration task will be quicker as only modified files must be treated.
If the source is still used by users in production during migration, more migration runs will be needed to transfer files modified by users since previous run to the source.
The next runs of the scanning can also be quicker if Miria supports FastScan on the source storage; it is the case for several storage models.
FastScan is based on Snapdiff techniques.