RCP and RAP applications are typically built using the Eclipse E4 Tool package. There may be test cases unique to the target underlying operating system -- Eclipse runs on OS X, Linux, Windows, and a few other OSes. First, get the Delta Pack -- http://download.eclipse.org/eclipse/downloads/drops4/R-4.4.2-201502041700/#DeltaPack
The Delta Pack contains all the platform specific resources from the SDK and is used for cross-platform exports of RCP applications. If you know how to install new software to Eclipse ("Help", "Install New Software", "Add"), then put the following into the Location field:
http://download.eclipse.org/e4/updates/0.17
After you have a stable platform and know how to create and modify RCP and RAP applications using E4 and the surrounding technology tooltips, you are on the way to know how to pentest these apps. Many will affect Eclipse only and few will call out to a file on the operating system or a network. If they do, you can leverage many existing Java-friendly pen-test tools such as Javasnoop, JDB, JSwat, DSer, untidy (for XML), et al.
The closest thing I can find to a RAP application would be something like Oracle Forms. Here is a blog post describing three ways to target an Oracle Forms app that uses HTTP or TLS. The blog does recommend Javasnoop for other more-flexible tasks. Here are additional use cases for Javasnoop:
For the non-Java portions of a RAP pen test, such as Javascript, you may want to look at retire.js (a tool similar to OWASP Dependency Check that can be used to analyze third-party Javascript components for well-known vulnerabilities) or Fortify SCA (a good securtiy-focsued static-analysis engine that works on Javascript code).