This chapter describes a template that can be used to estimate the system configuration for Spotfire Streaming applications.
Add up the following sizes to determine the minimum physical memory size required.
-
Shared memory size
-
JVM heap memory size
-
Spotfire Streaming Runtime executable size
-
Operating system size
-
Non-Spotfire Streaming process sizes
Example:
A Spotfire Streaming application using highly available objects with a single active node and a single replica node on separate machines.
Size of application highly available object - 4 kilobytes Number of highly available objects persisted - 1000000 4096 * 1000000 = 4096000000 bytes (approximately 4 gigabytes). Number of transactional objects modified per transaction - 4 Amount of memory modified per object - 8192 bytes Number of concurrent transactions - 64 4 * 8192 * 64 = 2097152 bytes Primary Machine: 70 Megabytes (Runtime shared memory footprint) 4096 Megabytes (highly available objects persisted in shared memory) 2 Megabytes (in flight transactions) 100 Megabytes (TIBCO Streaming process memory) =========================== 4268 Megabytes + JVM memory + Operating system memory + Other processes memory The backup machine has the same memory requirements as the primary machine.
Processing time for a unit of application work (e.g. request/response processing), less any time spent blocked waiting (e.g. Disk I/O, or waiting for a response from an intermediate system).
The elapsed time for servicing a unit of application work. This is the sum of the path length cost and any time spent blocked.
If the latency is larger than the required target then it can be reduced by one or more of the following items:
-
Decrease the amount of path length work (simplify or optimize the application).
-
Caching data in memory instead of retrieving it from an external resource (such as disk or network).
-
Increasing the performance of resources which are waited for (such as faster disk or network).
-
Increasing processor clock rate.
In an ideal system, with a parallelizable load, the message throughput is simply the number of processors divided by the path length.
When using High-availability, Managed object data is transparently copied across the network between nodes. The size of the object data copied is approximately the same size as the optimal allocationSpaceBytes reported by the memoryUsage() report shown earlier in the Shared Memory section of the Sizing Metrics chapter. There are also PDU headers sent.
-
Header — 28 bytes
-
SendMarshal — 112 bytes
-
ResponseMarshal — 56 bytes
-
TransactionInfo — 40 bytes
The following calculations may be used for estimating network traffic between each of the nodes that are part of the Managed object's High-availability partition:
For each Managed object created, there is a request:
Header + SendMarshal + ObjectDataSize
and a response:
Header + ResponseMarshal + ObjectDataSize + TransactionInfo
Example for the creation of a Managed object of 1024 bytes:
1024 + 28 + 112 + 1024 + 28 + 56 + 40 = 2312 bytes
For Managed object updates and Managed object multiple objects are sent in the same PDU, the request contains:
Header + SendMarshal + (N * ObjectDataSize)
where N is the number of objects modified in the current transaction. This is limited by maximumPDUSizeBytes, which defaults to 1000000. Larger transactions are spread across multiple network requests.
The response contains:
Header + ResponseMarshal + TransactionInfo
Example for the update of Managed object 1 (1024 bytes) and Managed object 2 (2048 bytes) in the same transaction:
28 + 112 + 1024 + 2048 + 28 + 56 + 40 = 3336 bytes
-
The space required by the Spotfire Streaming installation (typically around 2 Gigabytes).
-
If using file based shared memory, the space required by the shared memory file. This size is configurable and set at node installation time. System V shared memory does not use disk space for storing the shared memory.
-
Swap space on the system, at least the same size as physical memory.
-
Space for node log files. In typically production operation not much space is needed, but these files can grow over time, and can grow at faster rates if extra system tracing is enabled, or if there are many exceptions or deadlocks in the system. 1 Gigabyte should be a sufficient amount of space to start with.
-
Space for application logging. This is application dependent.