Support multiple subnets per AZ

[ashwanthkumar]

Most often than not, it is common (atleast among us) to share subnets across clusters than create 3-4 subnets for each application. Since each application's scale might vary we could co-host most of them under different ASGs but same subnet internally.

[brewkode]

I would rather have matsya detect subnets and see if there's IP ranges free inside them and then pick it. Having to configure the Subnet ID is a good start, but, doesn't lend itself well, when there's high spot fluctuations as most of the infra would get shuffled around. Having them fixed via configs, will make it ineffective. Thoughts?

[ashwanthkumar]

I would like the system to detect subnets in each AZ and pick one from the pool, but I guess that might cause more issues in terms of operations.

For example, consider a deployment pattern where Matsya is deployed along side Job Tracker across multiple Hadoop Clusters. It'll be very difficult to synchronize access between them. The way I see it, we need to build a global service on top of Matsya and Vamana and integrate these two. Something we don't have today.

[manojlds]

I propose this: Mark subnets with appropriate tags that identify the cluster that they belong to. The cluster name in the matsya config can be use for this purpose.