Monday, July 17, 2017

Improving Fortify Scan Time for Large Projects

If the project code base being scanned is in GBs, Fortify scan takes several days for scanning, irrespective of how powerful machine you are using for scan. Estimated times -

There are few ways to improve the scan time dramatically 'without compromising the scan coverage or breaking up the code base into smaller chunks':

Step 1:

While calling the sourceanalyzer utility, pass the parameters -
  • -Xmx<size>M or -Xmx<size>G
  • -Xss<size>M or -Xss<size>G
where -Xmx<size>M configures the max amount of heap memory and -Xss<size>M configures the maximum stack memory that JVM can use while running sourceanalyzer. 

You can assign the remaining memory for sourceanalyzer, after calculating the memory needed for OS and other running processes. The machine should be dedicated only for scanning and no other unnecessary (w.r.t Fortify scan) programs should run in the machine. 

As mentioned in HPE_SCA_Perf_Guide_17.10 - "Heap sizes between 32 GB and 48 GB are not advised due to internal JVM implementations. Heap sizes in this range perform worse than at 32 GB. Heap sizes smaller than 32 GB are optimized by the JVM. If your scan requires more than 32 GB, then you probably need much more than 48 GB such as 64 GB or higher."

Example: 
sourceanalyzer -Xmx4G -Xss1G -b "MyBuild" -cp "path/to/class-file" "path/to/code"
sourceanalyzer -Xmx4G -Xss1G -b "MyBuild" -scan -f "path/to/fpr"

If you set SCA_VM_OPTS environment variable, the same effect can be achieved without passing the memory parameters to each sourceanalyzer call.

Step2:

Enable parallel analysis in one of the following ways:
  • Add the -mt option to the analysis phase command-line invocation. For example: sourceanalyzer -b MyBuild -scan -mt
  • Add the property setting com.fortify.sca.MultithreadedAnalysis=true to your fortifysca.properties file, located in the <sca_install_dir>/core/config directory.
If you need to restrict the number of concurrently executing threads, you can set the
com.fortify.sca.ThreadCount property setting in the fortify-sca.properties file to the
number of available processor cores. By default, Fortify uses all available cores.

NOTE: the parallel analysis is only effective during scan, not during build. This options are available ONLY from SCA 17.10 onwards. earlier versions used the -j option along with the scan command to achieve the same.

Reference: HPE_SCA_Guide_17.10 in Fortify documentation.

4 comments:

  1. Just wondering..How much time does it save if we configure these options? Say a job runs for 6 hours or so..how much does it come down to If I configure these?

    ReplyDelete
    Replies
    1. I didn't do any concrete evaluation of the time saved. But I have seen some projects with code size 2-4 GBs running for 4 days and QE team terminating them. Using the multithread mode, the same scans were completed within a day.

      Delete
  2. Nice blog... Thanks for sharing steps to improve fortify scan time. This information on Micro focus fortify sca very helpful.

    ReplyDelete

Proxychain tool in a nutshell

In order to do penetration testing anonymously and decrease the possibility of identity detection, hackers need to use an intermediary mach...