Adjustments for Large Deployments / Best Practices

[w3ttr3y]

This merge request addresses issues #5 and #6 by adding and using two macros.

mhn-index macro

This addresses #6
While this macro is blank by default (to preserve backwards compatibility and because I would argue that's a best practice), this macro can be set by Splunk administrators to limit the searches to an index or indexes.

In its most basic usage, an administrator would likely create a file local/macros.conf with:

[mhn-index]
definition = index=mhn
iseval = 0

In more complicated situations, an administrator could include multiple indexes:
[mhn-index]
definition = index=mhnA OR index=mhnB
iseval = 0

Finally, in multi-tenant instances of Splunk, an administrator can override this macro from outside the applications.

For example, I could make two apps TA-mhn-companya and TA-mhn-companyb and I could limit access to them to two roles companya and companyb respectively. In the apps, I could set mhn-index to index=companya and index=companyb. By doing this, when a user logs in with the role companya, they can load the exact same dashboards, but they will only see companya's data and the filtering will be done in an efficient way. This small change really transforms this app into being multi-tenant ready.

mhn-base macro

Justification for mhn-base macro

All of the searches started with the a common root of source="mhn-splunk.log" If you apply basic software engineering principals, if you're repeating yourself frequently, consider factoring it out. In Splunk apps, that best way to do that is with a macro (yes saved searches would seem to be a good answer, but we can efficiently filter with saved searches without jumping through hoops)

Justification for mhn-base macro's default value

Default value: ``mhn-index sourcetype=mhn-splunk* OR sourcetype=mhn

1. \mhn-index`` is leveraging the index macro referenced above
2. sourcetype=mhn-splunk* this transitions from using source to a sourcetype. While existing administrators may have set different sourcetypes (since I haven't found any recommendations for one), the sourcetype that Splunk would have chosen is mhn-splunk, mhn-splunk-1 if the sourcetype learning felt the logs were different enough they were actually different types of logs, or mhn-splunk-toosmall if there weren't enough entries for the automatic sourcetyping to profile the logs By using a wildcard at the end, we will match all of these sourcetypes. This method also has the advantage of avoiding the wildcard at the begining like the previous source did which will provide a noticable performance improvement in large Splunk deployments.
3. sourcetype=mhn - in my opinion, the most obvious sourcetype for the data is mhn. A best practice would then be to set sourcetype=mhn on the inputs; this is included in the macro so that if best practices are followed, the logs will show up.

An alternative to using sourcetypes could be modifying the source argument to be /var/log/mhn/mhn-splunk.log but the typical Splunk way would be to use sourcetypes.

Misc Changes

My apologize on this part; I was cleaning up the app for deployment and removed the empty default/savessearches.conf and metadata/local.meta
I would have prefered to address these issues as a separate merge request, I think this is for the best, so I'm being lazy and not re-doing the merge request.

savedsearches.conf

Why include an empty file? Initially I checked it to look for scheduled searches (performance impact), searches that are set to run as owner instead of user (potential security issues), and to see what the dashboards may have (they are using inline searches which is what I would generally recommend)

local.meta

Ideologically: When apps are distributed, they shouldn't have anything in local (or metadata/local.meta) as that is reserved for local administrators/users to use. This should prevent app updates from being approved on Splunk base

Practically:
In default.meta, only admin and power users have write permissions in the app
[]
access = read : [ * ], write : [ admin, power ]

lines 146-147 of local.meta changed this to give everyone write access to the app

lines 13-18, 111-144 give everyone write access to a number of views (dashboards)
I wouldn't give people write access to the dashboards to preserve what is there by default. People can always clone and then modify the dashboards if they want/need customizations.

lines 20-109 referenced saved searches which were not in the app; I would consider it cruft that should be cleaned up

[w3ttr3y]

My apologizes are my text has some formatting issues; I need sign off for tonight, so hopefully they aren't terribly distracting.