Cross file analysis ?

[Gby56]

Hey ! Just wondering if you had the problem of performing the detection cross-file ?
Like, one index file, and many controller files with functions declared there to handle your request, parameters, response etc...
I know there are not a lot of tools doing that properly for now, Noir is purely regex based and not able to do it cross-file.

Here with semgrep I'm guessing you're limited with the interfile feature that is for the pro engine only ?

[Gby56]

My bad, I just noticed you talk about Interprocedural in your paper, I think that's what it is, my use-case for inter-file (as semgrep calls it) is to be able to really track down the request parameters, and the response structure, to generate accurate OpenAPI Specifications.

[mschwager]

Hey!

Yup, interprocedural / interfile analysis is something route-detect is currently weak at. I think Semgrep's interprocedural support could help, but as you mentioned, it's Pro-only, and I haven't had a chance to test it out. There are a few common web application framework patterns that route-detect has problems detecting authn/authz information:

• Controller + view. In this situation the authn/authz information is typically specified on the controller, and the route information is specified on the view. These are often separate files, so analysis is difficult. Rails is a good example: Improve Rails support #8. Django is similar. I tried some experimental analysis of these types of frameworks with the --interprocedural flag here:

route-detect/routes/main.py

Lines 74 to 78 in ac74a05

	viz_parser.add_argument(
	"--interprocedural",
	action="store_true",
	help="Expiremental: enable interprocedural security configuration detection",
	)

• Global configuration. In this situation the authn/authz information is specified in some global configuration object and routes are specified elsewhere in the codebase. This is basically the interprocedural case, but it's one-to-many instead of many-to-many. The authn/authz analysis is different, but you can think of it in that way. Many of the Java web app frameworks fall under this pattern. Again, I tried some experimental analysis with the --global flag here:

route-detect/routes/main.py

Lines 68 to 73 in ac74a05

	viz_parser.add_argument(
	"--global",
	dest="_global",
	action="store_true",
	help="Expiremental: enable global security configuration detection",
	)

• Middleware-based frameworks. These are common in Go and JS frameworks. Middleware approaches aren't inherently a problem, but often the authn/authz middleware is configured in a different location than the route information. Typically the middleware can be, and often is, close to the routes, so analysis is easier, but it can also be logically far from the routes. I didn't do any deeper analysis with experimental functionality here, but it's something to be aware of.

---

Most generally, all these cases fall under interprocedural or interfile issues. Generally Semgrep is weak in this situation, and something like CodeQL would be much stronger. However, CodeQL is significantly more difficult to use, and it'd take quite a long time to get the equivalent level of web application framework coverage that route-detect currently has.

[mschwager]

Hey @Gby56,

I recently added CodeQL support to route-detect, which excels at interprocedural/cross file analysis. It currently only supports Rails, but if you'd like to implement your own CodeQL queries, then the viz functionality will support any CodeQL SARIF output. I haven't released the new functionality to PyPI yet, but if you'd like to download the repo and beta test the functionality it's now available 😄