Faster Reloads With “Core Control”

QlikView is multi-threaded in many areas but not during the load process. Next time you’re running a load script, open the Windows Task Manager. If you have 2 cores, the processor usage will never go above 50%. You can to do this while QlikView is running and see what actions are single- and multi-threaded.

Load times on large datasets can be dramatically improved by splitting the reload process into pieces and assigning pieces to separate cores. A good place to start is monitoring how your Windows installation allocates processes to cores. I’ve only seen one tool that shows processes on each core. If you know of any others, please post a comment.

• CPU-Control

Now that you have evaluated the automatic allocation, how can you specify the core directly? One tool that ships with Windows is the Task Manager. Unfortunately, it can only alter a running process and it does not save your configuration for the next run. NOTE: Vista allows command-line affinity setting and Server 2008 might as well.

There are several choices of utilities to control core affinity. Some are command-line such as BeyondLogic’s, others have a GUI and store your choice for the future such as Tom’s Hardware Guide’s. The downside with THG’s tool is that you will need to make copies of QV.exe with different names so each gets a unique process name. The upside is that the CPU affinity setting is automatic.

• BeyondLogic Process Utility
• Tom’s Hardware Guide Affinity Tool
• BindCPU Utility
• ProcAff
• SysInternals’ PsExex (written for remote execution)
• ImageCFG.exe from an older Windows Toolkit

Obviously, trying to do this with Publisher would be a challenge. The upcoming Publisher 8.5 will support command-line execution which may work with these utilities, however you will lose visibility in the Publisher console. It’s all tradeoffs, but at least CPU affinity is another tool in your belt.

CPU Control

  Tom’s Hardware Guide Task Assignment Manager

Related posts:

1. QlikView Scalability