Tensor Parallelism Support

[intelava]

Thanks for this repo. It is very useful. But one thing, do we need to add support for tensor parallelism? As it changes the input shapes to the kernels. Actually I implemented that on my local. If you want I can create a PR.

[kyr0]

I'm not involved here yet, really, but def. PR, I'd say :) Sounds really cool what you've implemented locally!