Failure to update configuration with multiple dependent controller services and processors #39
Description
Looks like there has been a regression in version 1.1.32 (ce60cb1) when updating a controller service with a dependent processor, when said controller service is itself dependent on another controller service (i.e. processor 1 -> controller service 2 -> controller service 1, where -> is a dependency). This appears to have originated in commit 1c475d5. Following the steps from issue #35 now results in:
$ java -jar nifi-config/target/nifi-deploy-config-*.jar -m updateConfig -b "root > controller_service_dependencies" -c controller_service_dependencies.json -n http://localhost:8079/nifi-api
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.Main:109 - Starting config_nifi 1.1.32-SNAPSHOT on mode updateConfig
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.Main:132 - Communicate with nifi 1.3.0
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:66 - Processing : controller_service_dependencies.json
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - GetFile (33b7ba02-6f96-3915-38aa-a9c999512279) is already STOPPED
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) is already STOPPED
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:76 - [root, controller_service_dependencies] is stopped
2018-01-23 09:57:23 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:288 - Update config processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:320 - Processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) have config updated
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:288 - Update config processor GetFile (33b7ba02-6f96-3915-38aa-a9c999512279)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:320 - Processor GetFile (33b7ba02-6f96-3915-38aa-a9c999512279) have config updated
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) is already STOPPED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:288 - Update config processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:320 - Processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) have config updated
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:211 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) is replaced by (3efc8893-ab75-35d9-2e66-82ea3ef8ac85)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) is already STOPPED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) update for DISABLED
2018-01-23 09:57:24 | INFO | [pool-5-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) waiting for DISABLED
2018-01-23 09:57:24 | INFO | [pool-5-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) is DISABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:76 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) is UPDATED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:76 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) is UPDATED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:211 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) is replaced by (8d17b40b-5cd3-3134-da58-49dbb5c2f58e)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) is already STOPPED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:288 - Update config processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:320 - Processor ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) have config updated
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.UpdateProcessorService:211 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) is replaced by (7db4fe70-133f-3576-fc87-0e015daa8412)
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ProcessorService:56 - ConvertRecord (55f2cf94-a952-343a-5c72-471ed80adcfd) is already STOPPED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) update for DISABLED
2018-01-23 09:57:24 | INFO | [pool-12-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) waiting for DISABLED
2018-01-23 09:57:24 | INFO | [pool-12-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) is DISABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:76 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) is UPDATED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) update for DISABLED
2018-01-23 09:57:24 | INFO | [pool-13-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) waiting for DISABLED
2018-01-23 09:57:24 | INFO | [pool-13-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) is DISABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:214 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) trying removing
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:216 - JsonRecordSetWriter (3efc8893-ab75-35d9-d2d6-d1c247ac4aab) is removed
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) update for DISABLED
2018-01-23 09:57:24 | INFO | [pool-14-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) waiting for DISABLED
2018-01-23 09:57:24 | INFO | [pool-14-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) is DISABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:214 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) trying removing
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:216 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9125-493416ff8f6f) is removed
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) update for DISABLED
2018-01-23 09:57:24 | INFO | [pool-15-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) waiting for DISABLED
2018-01-23 09:57:24 | INFO | [pool-15-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) is DISABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:214 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) trying removing
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:216 - CSVReader (7db4fe70-133f-3576-945d-f720958368db) is removed
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9f89-ea8075fffac6) update for ENABLED
2018-01-23 09:57:24 | INFO | [pool-16-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9f89-ea8075fffac6) waiting for ENABLED
2018-01-23 09:57:24 | INFO | [pool-16-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - AvroSchemaRegistry (8d17b40b-5cd3-3134-9f89-ea8075fffac6) is ENABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) update for ENABLED
2018-01-23 09:57:24 | INFO | [pool-17-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) waiting for ENABLED
2018-01-23 09:57:24 | INFO | [pool-17-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - JsonRecordSetWriter (3efc8893-ab75-35d9-2e66-82ea3ef8ac85) is ENABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - AvroSchemaRegistry (8d17b40b-5cd3-3134-da58-49dbb5c2f58e) update for ENABLED
2018-01-23 09:57:24 | INFO | [pool-18-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - AvroSchemaRegistry (8d17b40b-5cd3-3134-da58-49dbb5c2f58e) waiting for ENABLED
2018-01-23 09:57:24 | INFO | [pool-18-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - AvroSchemaRegistry (8d17b40b-5cd3-3134-da58-49dbb5c2f58e) is ENABLED
2018-01-23 09:57:24 | INFO | [main] c.g.h.n.c.s.ControllerServicesService:110 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) update for ENABLED
2018-01-23 09:57:24 | INFO | [pool-19-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) waiting for ENABLED
2018-01-23 09:57:24 | INFO | [pool-19-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) is ENABLING
2018-01-23 09:57:26 | INFO | [pool-19-thread-1] c.g.h.n.c.s.ControllerServicesService:114 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) waiting for ENABLED
2018-01-23 09:57:26 | INFO | [pool-19-thread-1] c.g.h.n.c.s.ControllerServicesService:116 - CSVReader (7db4fe70-133f-3576-fc87-0e015daa8412) is ENABLED
2018-01-23 09:57:26 | INFO | [pool-21-thread-1] c.g.h.n.c.s.ControllerServicesService:139 - 3efc8893-ab75-35d9-d2d6-d1c247ac4aab cannot be enabled because it is not valid: '' is invalid because Failed to run validation due to java.lang.IllegalStateException: Could not find any Processor, Reporting Task, or Controller Service with identifier 3efc8893-ab75-35d9-d2d6-d1c247ac4aab
Exception in thread "main" com.github.hermannpencole.nifi.config.model.ConfigException: com.github.hermannpencole.nifi.swagger.ApiException: Conflict
at com.github.hermannpencole.nifi.config.utils.FunctionUtils.runTimeout(FunctionUtils.java:40)
at com.github.hermannpencole.nifi.config.utils.FunctionUtils.runWhile(FunctionUtils.java:65)
at com.github.hermannpencole.nifi.config.service.ControllerServicesService.setStateReferencingControllerServices(ControllerServicesService.java:127)
at com.github.hermannpencole.nifi.config.service.UpdateProcessorService.updateControllers(UpdateProcessorService.java:176)
at com.github.hermannpencole.nifi.config.service.UpdateProcessorService.updateByBranch(UpdateProcessorService.java:88)
at com.github.hermannpencole.nifi.config.Main.main(Main.java:137)
Caused by: com.github.hermannpencole.nifi.swagger.ApiException: Conflict
at com.github.hermannpencole.nifi.swagger.ApiClient.handleResponse(ApiClient.java:1104)
at com.github.hermannpencole.nifi.swagger.ApiClient.execute(ApiClient.java:1027)
at com.github.hermannpencole.nifi.swagger.client.ControllerServicesApi.updateControllerServiceReferencesWithHttpInfo(ControllerServicesApi.java:1011)
at com.github.hermannpencole.nifi.swagger.client.ControllerServicesApi.updateControllerServiceReferences(ControllerServicesApi.java:996)
at com.github.hermannpencole.nifi.config.service.ControllerServicesService.lambda$setStateReferencingControllerServices$1(ControllerServicesService.java:137)
at com.github.hermannpencole.nifi.config.utils.FunctionUtils.runWhile(FunctionUtils.java:54)
at com.github.hermannpencole.nifi.config.utils.FunctionUtils.lambda$runWhile$0(FunctionUtils.java:65)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)
Perhaps this issue was always present, but only noticeable now since nifi-config attempts to limit the scope of controller services it enables/disables in 1c475d5?