GNIP 103 - Removal of the Advanced Workflow from GeoNode

[giohappy]

GNIP 103 - Removal of the Advanced Workflow from GeoNode

Overview

The Advanced Workflow is a generic term that refers to a mix of options that allows controlling approval and publishing workflows, and connected permissions, when activated.

This was introduced long ago in GeoNode, and it went through several changes and refactoring during the course of GeoNode releases. During this process, a list of idiosyncrasies has been introduced, leading to very complicated operations that depend on several interrelated states and configurations. These operations also required the introduction of transition flags inside the resource model (was_approved, was_published, etc.), which highlight the brittle nature of the flow.
Reasoning about what happens to permissions and publishing status of a resource when one or multiple Advanced Workflow options are activated is a complex task, with many side effects on user permissions.

For all these reasons, the GeoNode team has considered the refactoring of the AD model several times. In the end, the decision to remove it was taken to give space to a more granular control supported by the introduction of the Permissions Registry. With this component, we can tune permissions granularly and without having to translate them into Guardian (DB) permissions. We can perform ACL logic based on business rules, roles, and configurations in a cleaner and clearer way.

Proposed By

Giovanni Allegri - GeoSolutions srl (@giohappy )

Assigned to Release

This proposal is for GeoNode 5.

State

• Under Discussion
• In Progress
• Completed
• Rejected
• Deferred

Motivation

The ADW is a mix of settings that enables a mix of interrelated configurations, which activate a list of behaviors that affect resource states and permissions.
These settings have changed and undergone refactorings during the course of GeoNode's history. Some of these changes have introduced exceptions and specific cases that have led to a very complicated logic within the permission systems and the resource management, spread across many places of GeoNode's source code.

Efforts have been made in the past to limit the number of points where ADW logic is enforced, but still, the logic is not flexible and mixes aspects that should be managed orthogonally instead. For example, some of the settings affect resource visibility but also permissions on the resource, which involve group membership of the resource and of its owner, without fine-grained control to turn on and off single options. Thi makes the ADW very specific and unflexible.

We don't think the ADW as it is now is really useful and understood by users and admins.
Simpler approaches have been proposed in the past, but there hasn't been the opportunity yet to transform the ADW into a set of independent and granular controls, with a clear separation between resource visibility, approval, and permissions.

A big effort has been made to refactor the permissions system and collect the ACL logic under a central module (the Permissions Registry) and introduce the concept of "virtual" permissions. This move couldn't be completed yet because of the complications in the ADW code and logic.

We propose to remove the ADW completely and give space, if needed, to a more lightweight and cleaner approval workflow in the future.

Proposal

The ADW code will be removed from GeoNode along with the settings that control it:

• RESOURCE_PUBLISHING
• ADMIN_MODERATE_UPLOADS

GROUP_PRIVATE_RESOURCES won't be removed since it doesn't affect only the ADW, but also logic outside of it.

The was_approved, and was_published ResourceBase fields will be removed.
The approved, published, ResourceBase fields will be deprecated. They will be removed in later minor versions.

Backwards Compatibility

This is a breaking change. Users relying on the publishing and approval flows will lose this functionality.
Permissions that have been set by the ADW logic won't be lost with the upgrade to GeoNode 5. They will remain inside the permissions DB, but resource owners will gain full permissions on the resource.

Future evolution

Approval workflows might be introduced again in the future, if it will be requested, but with a cleaner and simpler design.

Feedback

Update this section with relevant feedback, if any.

Voting

Project Steering Committee:

• Alessio Fabiani: +1
• Francesco Bartoli:
• Giovanni Allegri: +1
• Toni Schoenbuchner:

Links

Remove unused links below.

• Email Discussion
• Pull Request
• Linked Issue

[afabiani]

+1 here

[kikislater]

I am not involved in the project, but in my opinion, this is Geonode's main feature for controlling users, groups, and datasets. Does this mean that the four possible workflows will be removed? Automatic publishing, simple publishing, advanced workflow, and simplified workflow as well? Even though there are some strange things in the advanced workflow, for example with anonymous inherited rights, and we have made some changes on our side, it is a valuable customization.
So, in version 5, if I understand correctly, there will only be the automatic publishing workflow, without granularity to control resources and groups. What is the plan to cover the approval workflow in the future?

[mwallschlaeger]

Hi, we are currently a running our geonode instance with the advanced workflow activated. I could image the modify our workflow to also work without the ADW, but what we are really missing is a permission system to give permission to forbid uploads by all users accepts granted user. And further have a state, where datasets are not published before reviewed by a "staff" or "admin" role person. Our usecase might be relevant for more than just our scientific usecase.

[ridoo]

We also have multiple projects running, where uploads have to be reviewed before being published. However, I agree with the actual problem statement, that the current setting foo became confusing and opened the door for some implementation quirks.

[giohappy]

Thanks to all for your comments.
I missed an important part (thanks @Jo-Schie for the hint) of the description because I was still elaborating on it. I will say it briefly here, then I will integrate the GNIP.

The idea is, IMHO, in line with your comment: rather than performing very obscure state transitions, automatic assignments, and implicit rules, we want to enhance the management of permissions (and the management of permissions to grant permissions) so that it will be more granular, adaptable, and clear.

Some of the things we're considering:

• Start using the "staff" role, which is already there but never used. Initially, it should be an admin without access to the Django Admin and Geoserver.
• Implement support to configure who can assign the permissions for the Public and Registered Users groups. By default, it will be the owner + admin/staff, but it could only be admin or admin + staff

The proposal is to remove the "approved" and "published" flags, because they wouldn't have any hidden effect. Visibility will be controlled explicitly through permissions. If only the admin/staff can give public permissions on a resource, here we have the basic ADW functionality.

We don't have state flags anymore, but do we really need them? It would be tricky to update their values automatically. What (clear) logic should drive it?
And having flags to be updated manually.... Well, in my experience, nobody will update them. But maybe you have a different experience.

[ridoo]

Well, to me having a published flag indicates more to a certain state in a publishing workflow (which goes beyond just uploading data). In my experience, this is something which is kind of expected when operating a data portal letting users upload their data. You in control want to review the content before making it available. In the end (technically spoken) it will be granting or not granting access to a dataset. However, the published or approved status could be made explicit to indicate a dataset gained such a review and is ok to be accessible. I think, it would be also helpful for people having the task to review new uploaded data to filter according to not approved yet.

Just wanted to add my quick thoughts on this. Perhaps, other who are actually operating GeoNode and having such workflows requirements can comment on this as well.

/cc @mwallschlaeger
/cc @Jo-Schie
/cc @vineetasharma105

[Jo-Schie]

To me, the current implementation of the Advanced Workflow seems to be "half backed" and not user friendly at all. It's impossible for me to teach people to use it and I would also not expect people to use and update flags without a proper workflow implementation. Maybe if you only use the GeoNode to publish selected datasets and if you have an expirienced, specialized team that checks the GeoNode every morning when trinking their coffeees... but "normal" or "first-time" users are totally confused with the present user expirience.

Current state is: The flags are hidden in the metadata and users are not notified via Email or UI about

• actions they have to take (e.g. review or publish)
• about changes in the state of their ressource ("was reviewed ", "was published ").
• about who is making changes or supposed to make changes ("needs review from A", "needs approval from B")
• the current state of the ressource ("is published", "is reviewed").
• all of the permissions and group assignements that affect the ressource.

My understanding was, that implementing a proper notification and workflow stream would be a much larger task and much more difficult to achieve. I am not against such a workflow, but it should be implemented properly and not managed with checkboxes and flags.

I think developing such a workflow should be done in an agile mode. First step would be to cleanup, centralize and facilitate the "normal" sharing actions, so that everything is in one place. This is what @giohappy is proposing and I'm happy with that. Second step could be to conceptualize and redesign a publication workflow that is embedded in the sharing panel to have a simpler user expirience.

In the meantime I think most users will be totally fine to inform their peers via email about ressources they would like to have published. Looking at the GeoNode community most institutions who publish data seem to be small teams where direct communication is happening propably anyways on a daily base. So I don't think that the flags will be hardly missed.

[ridoo]

I think developing such a workflow should be done in an agile mode. First step would be to cleanup, centralize and facilitate the "normal" sharing actions, so that everything is in one place. This is what @giohappy is proposing and I'm happy with that. Second step could be to conceptualize and redesign a publication workflow that is embedded in the sharing panel to have a simpler user expirience.

Totally agree here. Just wanted to make point out, that there is a difference of granting access to data and "publishing data" as a process. Just removing things would fall short IMO. If cleaning up is the first step towards a redesigned, more explicit workflow, I am just fine with that.

[giohappy]

@ridoo yes, cleaning is certainly a required step, because the ADW and all the connected flags have piled up over the years, leading to an extremely complicated mix of behaviours and (often hidden) transitions. All backed by spaghetti code that we're fighting to bring back to something manageable :)

We might open a discussion on the topic, and clarify, for example, what "publishing" and "moderating" mean in your daily experience.

In this moment, the publishing and approved flags trigger automatic changes in permissions. That's it, basically. And these changes are not easy to understand. What happens is driven by a mix of settings that often affect each other in unexpected ways.

Our goal here is to clean things up while still giving users and admins control over who can change permissions on what.
The end result is not much different from what the ADW does. The main difference is that, after this change, you will have direct control over those permissions, instead of relying on automations without much control.

I agree, this doesn't replaces a "workflow". However, after this refactoring we should have a more "green field" where a proper workflow could be built, with strong semantics.