[FEATURE] When Kyuubi is restarted, handling YARN orphaned applications

[alan3879]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the feature

After changing Kyuubi configurations and restarting, the existing YARN applications are not restarted but continue to be assigned to the existing applications. While it is not possible to explicitly terminate these applications, orphaned application issues seem to occur, and a solution for handling them has not been found.

It would be ideal if the Kyuubi server could terminate the applications it created upon shutdown and restart them when starting up again

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

• Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
• No. I cannot submit a PR at this time.

[github-actions]

Hello @alan3879,
Thanks for finding the time to report the issue!
We really appreciate the community's efforts to improve Apache Kyuubi.

[pan3793]

it's by design, that the engine(in your case, spark app runs on YARN) and server manage each lifecycle by themselves, in this case, you should kill the app manually.