Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52870

Make Zookeeper optional for ad hoc topics

    Details

    • Similar Issues:

      Description

      We may add another option which works without Zookeeper.

       

      One suggestion I have is that you might want to remove the requirement of connecting to ZooKeeper, which transitively means you can’t create topics on the fly. Many people, me included, will have locked down Kafka instances with zk totally hidden from us, so creating ad hoc topics just isn’t possible.
       
      After spending almost no time looking at it, I’m guessing you could set a header to whatever you had the ad hoc topic named and then do a if (header[‘agentName’] != ‘myAgentName’) return;
       
      If you’re worried about it being too chatty, you could shard it and create topics jenkins-remote-01 -> jenkins-remote-99 and then hash the agent name into a number to determine which topic to use (you’d still have to filter, but you’d have significantly fewer “false messages”).
      

        Attachments

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              pvtuan10 Pham Vu Tuan
              Reporter:
              pvtuan10 Pham Vu Tuan
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: