Is it a good approach to keep my automation system proprietary while clients use it under retainer, rather than handing over the system?
No, I don't think holding onto the system IP makes sense. We're past the point where a business can be built on that kind of intellectual property—your N8 nodes are just AWS Lambda calls chained together, nothing inherently proprietary. In a year or two there will be many alternatives, so holding on is pointless unless you're curing cancer. Instead, give clients full access to the IP and build everything in their account; that eliminates the handoff step and can increase conversion.