How Miria distributes Jobs to Datamovers 

Typical Miria configuration uses 1 database server and 1 or more datamovers (or agents). 

Pools of 1 or more datamovers can be created 

Excluding database server from pools avoids the risk of overloading it as the database server is also an agent (datamover). 

The database server records all required information to keep track of migrated files but also orchestrates the migration task(s) and their related job(s) 

 A migration task transfers files from source storage to destination storage (many types supported by Miria) 

 Per default, 1 migration task is using 1 job that is using 1 thread. 

But you can configure the task to run more than 1 job. 

And/or you can configure jobs to run more than 1 thread. 

 Job load balancing 

Starting the migration task will start scanning the source and the first job starts transferring data soon after. 

Then, if desired, more jobs can be fired based on criteria below that can be all defined: 

• Total size of files scanned 
• Time the scan job is running 
• Number of files scanned 

 As soon one of the criteria is reached, a new job is fired and Miria DB server assigns that new job to the datamover in the pool having the least number of jobs already running.  

Miria load job balancing criteria is based on the number of job(s) each datamover is running. 

 Single thread or Multithread 

Per default, 1 job is single thread consuming 1 CPU core and some memory. 

But 1 job can also be multithread in what case each thread uses 1 CPU core and some memory. 

The number of threads a job can run is a task’s configuration parameter. 

Based on that explanation, you can partially understand how a Miria configuration is sized. 

Take an example:  

1. Imagine a configuration with 1 Miria DB server and 1 Miria datamover where only the datamover will be used to migrate data (by using pool). 
2. The datamover server has a single 8 cores CPU. Let’s say we have enough RAM for this example. 
3. You can then run 8 single threaded jobs or 4 dual threaded jobs or 2 jobs with each 4 threads or 1 job with 8 threads.

Trying to run more jobs or more threads per job will cause performance problem! 

If you need more jobs/threads to complete the migration in the timeframe foreseen, you can add more datamover(s) so more jobs/threads can be spread across all of them. 

Performance will also be impacted by many other factors like the source and the destination storage’s load capabilities and the network path performance for instance.

The first migration’s run last the longest because : 

1. the first scan job must scan the entire source storage
2. the first run of the migration task(s) must transfer all files to the destination

The next run of the migration task will be quicker as only modified files must be treated. 

If the source is still used by users in production during migration, more migration runs will be needed to transfer files modified by users since previous run to the source. 

The next runs of the scanning can also be quicker if Miria supports FastScan on the source storage; it is the case for several storage models.
FastScan is based on Snapdiff techniques.