Feature Request: Static Config File

I have been attempting to use a configuration management tool to maintain my USB server configuration, but it's nearly impossible due to the fact that the server completely re-writes the config file every time it boots. Could you do something to mitigate this? To get things started, a few suggestions:

* Make it only append to/edit and not overwrite the file completely
* Store the 'it' value (which I'm guessing is the main reason it does this) elsewhere
* Allow value overrides using a '.d' type of directory. (I can elaborate further if I need to, not sure that's a good description)

#2

How does your tool manager the config? It writes meta-tags or something to the file so it know the location of things?

#3

I'm using SaltStack (http://saltstack.com/) to do it.

The way it works is that it generates the file based on a template and data that gets provided to the file template. It uses this generated file to create a diff against the file on-disk. If the diff shows that there are differences, it overwrites the old file with a new one and then restarts the daemon.

As a result of this sequence of events, it causes it to see the file as requiring changes every time, because the config file changes the second the daemon is restarted.

That all make sense?

#4

OK, actually ive changed the configuration management in the next version of the server 2.4.5 to be transactional and it will make external file management of the configuration impossible while the server is running. (When the server is not running you can change the config file with no issues)

The server will check every 5 seconds if the internal config state has changed if so the state is flushed to disk, similar to the way a database works.

The reason i made this change is that the client is going to be able to do more real-time configuration of the server (without having to edit config files and without a restart as is currently required) . I have quite a few customers wanting this and its useful for a lot of other things where ssh access is difficult or not available on the server.

I think to manage config files you can only do this when no servers are running on your devices. Then you can compare/copy between them. However when the config is flushed it disk, the order the settings are written to the file may change so im not sure that is viable for your use case...

#5

Unfortunately, I think you are correct that what you are speaking of is not really viable in my use case, as I don't think I have that much control over how and when Salt does its comparisons.

I struggled with what to say in response to the changes you're describing, and have settled on a conservative response, as I don't want to overstep the boundaries of your good will. I have some... very large... concerns about the changes you describe from both a user experience standpoint, as well as from a technical/engineering standpoint. That said, I do not want to assert my opinion where it is not welcome. If you would like my feedback/commentary on this aspect (coming from a software engineer with a decent amount of experience) I would love to go into detail, but will not put it out there unless you want it.

What I will do is put forth suggestions about two adjustments that would fix my very specific problem with minimal effort:

  1. Find a way to not need to store the 'it' value in the configuration. Seeing as it's just a unix timestamp (if memory serves correctly, at least) it seems very silly to store at all, much less in the configuration file. I mean, I don't really care about this much, except that so long as that property remains a part of the configuration, it means that you must rewrite the configuration file any time the server is booted/shutdown, even if there are not actual important changes to the configuration such as a license or port number change.
  2. Add a start-up (command line) flag that instructs the server to not allow run-time configuration changes. This is a rather dirty solution, but it would do the trick.
#6

Actually i dont need to store the "it" timestamp, i just checked the code again. But I dont think that will help you though because the config will be rewritten in random order from server 2.4.5 onwards. The server is quite complicated code with many threads doing lots of different things simultaneously and hence i needed to migrate to a transactional configuration design to be able to correctly implement real-time server state changes to guarantee state-machine changes internally. In a (very) future build, perhaps one server can migrate itself to another server by transmitting its current state over the wire that would keep servers in sync...

#7

Is there a technical reason as to why the order of the configuration stanzas must be random? When it rewrites the config, does it do the entire file at once, or does it cut the old definition out and append the new one to the end? I ask because if you are writing the entire config at once, then one way you could guarantee consistency is to have it flush the configuration keys out in alphabetical order. Alternatively, if you cut out the old data and append the new data to the end, what is stopping you from instead inserting the new data into the position where the old data previously exist? Heck, if you wanted to go overboard, you could even combine the two solutions. :)

#8

The configuration is stored internally as a linked list where key=value elements are add/removed/updated atomically and this is flushed to disk periodically by iterating over the list while holding the lock. The list is not sorted for efficiency and simplicity reasons. Multiple instances of the the linked structure is used throughout the server to handling pretty much everything. The server doesnt know about the config.ini, only thats its a file it loads at the start and re-writes as efficiently as possible.