How Miria distributes Jobs to Datamovers
Last updated on 16 December 2021 Eric Collart Miria : How To ?
Articles / Posts
- Lina 6.1 Controlled Release is available on-demand
- PowerShell for Lina 6.0
- Advisory ID : LINA/ADE-2023-0002
- Advisory ID : LINA/ADE-2023-0001
- Time Navigator 4.6.9 Hyper-V Software Alert
- Tina 4.7.1 GA is available !
- Miria install error on a Linux server without X11
- HSS Replication requires attention when upgrading to 5.x
- LINA Agents 5.3.4
- Miria 3.15 GA is available
- Lina – EN (21)
- Miria – EN (8)
- Tina – EN (17)
Typical Miria configuration uses 1 database server and 1 or more datamovers (or agents).
Pools of 1 or more datamovers can be created
Excluding database server from pools avoids the risk of overloading it as the database server is also an agent (datamover).
The database server records all required information to keep track of migrated files but also orchestrates the migration task(s) and their related job(s)
A migration task transfers files from source storage to destination storage (many types supported by Miria)
Per default, 1 migration task is using 1 job that is using 1 thread.
But you can configure the task to run more than 1 job.
And/or you can configure jobs to run more than 1 thread.
Job load balancing
Starting the migration task will start scanning the source and the first job starts transferring data soon after.
Then, if desired, more jobs can be fired based on criteria below that can be all defined:
As soon one of the criteria is reached, a new job is fired and Miria DB server assigns that new job to the datamover in the pool having the least number of jobs already running.
Miria load job balancing criteria is based on the number of job(s) each datamover is running.
Single thread or Multithread
Per default, 1 job is single thread consuming 1 CPU core and some memory.
But 1 job can also be multithread in what case each thread uses 1 CPU core and some memory.
The number of threads a job can run is a task’s configuration parameter.
Based on that explanation, you can partially understand how a Miria configuration is sized.
Take an example:
Trying to run more jobs or more threads per job will cause performance problem!
If you need more jobs/threads to complete the migration in the timeframe foreseen, you can add more datamover(s) so more jobs/threads can be spread across all of them.
Performance will also be impacted by many other factors like the source and the destination storage’s load capabilities and the network path performance for instance.
The first migration’s run last the longest because :
The next run of the migration task will be quicker as only modified files must be treated.
If the source is still used by users in production during migration, more migration runs will be needed to transfer files modified by users since previous run to the source.
The next runs of the scanning can also be quicker if Miria supports FastScan on the source storage; it is the case for several storage models.
FastScan is based on Snapdiff techniques.