High throughput
It has always been a challenge to get data from imaging devices into the computer as fast as they are available. 20 megabytes per second was a challenge in the late 1990’s and early 2000’s. So much so that imaging systems were usually peripheral devices between the camera and the computer. Some frame grabbers existed, but the ones with the highest bandwidth had to acquire images to their own memory and perform some of the processing locally before sending the results to the computer.
Today’s computers can handle much higher bandwidths, in the gigabytes per second range. Cameras are now emerging that can produce data at these speeds as well. The problem now is not getting the data into the computer; being able to process the data and display the results as fast as it is available. If the processing is even slightly slower than the incoming data, a delay will result and continue to grow over time.
High throughput is not just the ability to get huge amounts of data through the computer. It is also being able to synchronize multiple hardware devices as tightly as possible. As throughput increases, the closer in time the software can be aware an event occurred on a device and the faster it can cause another device to respond. So the higher the throughput, the tighter devices can be synchronized with software. In the past, other hardware devices were developed that could perform the synchronization. Hardware synchronizations, however, are limited to specific devices and do not offer quick solutions as the requirements change. Software synchronizations, on the other hand, have no limits; systems designers can easily reconfigure software synchronizations with fewer costs. Software controlled devices makes a system much more agile.
Agile Automation tackles these issues by creating applications that can spin multiple threads designed to work simultaneously. Multi-threaded applications together with event-driven communications are able to respond quickly and move data through the system much faster. This throughput increased dramatically as CPU manufacturers began increasing the number processing cores. Graphical display manufacturers are now introducing processors with multiple cores as well. This technology has had another dramatic effect with throughput on an agile system.
High throughput requires an agile architecture capable of scaling itself to take advantage of the hardware available. Multiple threads are the first step to achieving this goal since they define functionality in finer granularity. Well-designed software can spread out its threads amongst multiple CPU cores and GPU cores. Increasing the number of cores, therefore, helps achieve higher throughput. The system architect accomplishes this task by the specifying the necessary hardware. This deems the application very agile and is the goal of Agile Development.
Multiple threads
Agile Automation has been developing software with multiple threads of execution since day one. In fact, our founder was able to create a pseudo multi-threaded system within the Windows 3.1 operating system during the early 1990’s (Windows 3.1 was a multi-tasking OS that used a single thread of execution). Machine control applications can benefit from multiple threads, even when they run on a system with a single processor. This is because most machines consume time while performing their tasks. This “time” becomes idle time for the computer system as it waits for the machine to complete its task (or even to respond to a query). A software system that spins multiple threads designed to run simultaneously will take advantage of this idle time, whereby increasing its throughput.
Multi-thread applications are a little difficult to troubleshoot, however. The timing between two threads changes from one execution to another. This causes problems that may only reveal themselves intermittently, depending on the phase of each thread relative to the others. Agile Automation has many years’ experience developing multi-threaded applications and has a great working knowledge on isolating problems that can occur in such a system.
Multiple CPU cores
CPU manufacturers introduced processors with a technology called hyper-threading in the early 2000’s. This technology allowed a single processor to execute two separate threads of execution simultaneously. Before hyper-threading, the two threads actually executed through a context switching mechanism that switched between threads allowing each one a small amount of time before switching to the next. The time allotted each thread was very short (in the microseconds), so it gave the illusion that each thread executed simultaneously. With a hyper-threaded processor, the two threads can truly execute simultaneously. Well, to a point. The hyper-threaded processors are actually taking advantage of unutilized resources built into the processor. When both threads try to use those resources at the same time, one will have to wait for the other to finish.
The CPU manufacturers later introduced processors with two physical cores. These separate cores allow the two threads to execute closer to simultaneous than a hyper-threaded processor simply because they actually use separate resources and are not just trying to utilize the resources better. Manufacturers later introduced processors with multiple hyper-threaded cores, whereby increasing throughput even further.
The multi-threaded applications developed by Agile Automation took advantage of these technologies without having to rewrite any software. The first hyper-threaded processors nearly double the throughput of those systems. Multiple core processors have increased throughput dramatically. Agile Automation was ahead of the curve due to their early practices developing multi-threaded applications.
Multiple GPU cores
Graphical display manufacturers have been developing processors with multiple cores for many years now. A few years ago, they introduced the ability for applications to use these cores for processing data other than graphical data. The advantage of using a GPU for processing is they move and process data extremely fast. They can do this faster than a CPU for a few reasons. A GPU sits on the display card along with all its resources. This allows manufacturers to design their GPU with a specific purpose using specific components. In addition, placing the GPU very close to its resources means shorter traces enabling higher frequencies, and thus, higher throughput. CPU manufacturers design processors for greater flexibility because it has to control multiple unknown devices using non-specific resources. Greater flexibility normally causes more overhead, and the resources require more wiring and longer traces, all equaling less throughput.
GPU manufacturers have also designed their GPU with many more cores than found in a CPU (hundreds, even thousands depending on the GPU). Applications that can decompose their functions with a fine granularity can spread the work across these cores and achieve much higher throughput. Agile Automation has been designing software with fine granularity for many years. Taking advantage of GPU cores from an existing application became a relatively simple task. Agile Automation will decompose future applications with as much granularity as possible, whereby enabling the application to scale itself depending on the GPU cores available for the ultimate agile experience.