#45 ✓resolved
Christopher Stawarz

Reconsider where to store configuration, data, and cache files

Reported by Christopher Stawarz | May 7th, 2010 @ 05:00 PM | in 0.4.4

Currently, MWorks uses the following directories for storing data files, configuration files, and cache files, respectively:

  • /Documents/MWorks/Data/
  • /Library/Application Support/MWorks/Configuration/
  • /Library/Application Support/MWorks/Experiment Cache/

However, this seems inappropriate, as it uses system-wide storage locations for files that properly belong to individual users. We should consider ways to remedy this.

Data files could go in ~/Documents/MWorks/Data/. Or, even better, the client could provide an interface that lets the user pick a save location at runtime.

Cache files should go in ~/Library/Caches/MWorks/. Alternatively, if we decide that all users really should share a single cache, we should use /Library/Caches/MWorks/.

Unlike data and cache files, configuration files (i.e. setup_variables.xml) are likely to be shared by multiple users, since they primarily describe the available displays. However, we should support per-user config files, in case a user wants to change their setup (e.g. to not use an external monitor during testing) without affecting others. To enable this, MWorks should first look for config files in ~/Library/Application Support/MWorks/Configuration/ before looking at the system-wide config dir.

Comments and changes to this ticket

  • David Cox

    David Cox May 10th, 2010 @ 10:08 PM

    • State changed from “new” to “open”
    • Assigned user set to “Christopher Stawarz”

    Agreed on all of these -- none of them were meant to be permanent resting places, they were just expedient places to put things until something better could be sorted out.

    Issues and other notes:

    1. Re Data: Implementing the interface is a tad fussy since you're navigating the server machine's filesystem from the client machine. But that's just fussy, not insurmountable.

    2. Experiment Cache could also go in /tmp or ~/tmp, or anywhere else that is volatile (I assume that /Library/Caches gets trashed on log-outs?). It shouldn't functionally matter if users share a cache or not, this stuff gets rewritten on each and every load.

    3. Re Configuration: No major objections on this, though this could end badly if multiple users are using the same account on the machine (which is pretty common in labs I've seen). Also potentially dangerous for users to be messing with their setup configuration in multiple places (e.g. if they forget that they changed it in one place and collect data that way). Life is full of such dangers, though, I guess, and people will just need to be careful.

    Incidentally, there is also an "include" directive in the xml that could be used to achieve similar effects from the client-side.

  • Christopher Stawarz

    Christopher Stawarz May 11th, 2010 @ 02:18 PM

    • Milestone set to 0.4.4

    Re Data: Implementing the interface is a tad fussy since you're navigating the server machine's filesystem from the client machine. But that's just fussy, not insurmountable.

    For 0.4.4, I'd be happy just to switch from /Documents/MWorks/Data to ~/Documents/MWorks/Data. We can add the ability to choose an arbitrary save location in a later release.

    Experiment Cache could also go in /tmp or ~/tmp, or anywhere else that is volatile (I assume that /Library/Caches gets trashed on log-outs?). It shouldn't functionally matter if users share a cache or not, this stuff gets rewritten on each and every load.

    /Library/Caches is persistent and is never automatically emptied (however, it is not included in Time Machine backups). If the stuff in Experiment Cache is only meant to persist for a single session, then it probably should go in a more volatile location. It looks like the right way to do this is to use a combination of NSTemporaryDirectory and mkdtemp to create a temporary directory.

    Re Configuration: No major objections on this, though this could end badly if multiple users are using the same account on the machine (which is pretty common in labs I've seen). Also potentially dangerous for users to be messing with their setup configuration in multiple places (e.g. if they forget that they changed it in one place and collect data that way). Life is full of such dangers, though, I guess, and people will just need to be careful.

    Would it make more sense to look for the config only in ~/Library/...? That way, there'd be one and only one place to look for config files, but different user accounts on the same machine could have different configs, and you wouldn't have to worry about some other use changing the config without you knowing.

  • David Cox

    David Cox May 11th, 2010 @ 02:42 PM

    For 0.4.4, I'd be happy just to switch from /Documents/MWorks/Data to ~/Documents/MWorks/Data. We can add the ability to choose an arbitrary save location in a later release.

    No objections here.

    It looks like the right way to do this is to use a combination of NSTemporaryDirectory and mkdtemp to create a temporary directory.

    Again no objections. Occasionally it is helpful to be able to find this directory for debugging to do a post-mortem on the stuff that ends up in this directory (this is where the server unpacks the experiment passed from the client), but this is more for us than end-users, so no real issue there.

    Would it make more sense to look for the config only in ~/Library/...?

    I think if you asked Jim, he'd say that he'd be happiest if users had no idea where that file lived. I can see this point of view; users can do themselves serious harm if they mess with that file and all of a sudden all of their stimuli weren't where they thought they should be for last two months of data collection. People who run labs live in fear of this kind of stuff. Obviously, it is worse if they mess up others as well, but there's a a mess that comes with having multiple, possibly different setup config files, and what happens when you add new users, etc. etc.

    Maybe having an additional/optional ~/Library version of the configs is the way to go, since it discourages casual users from touching the system-wide file and screwing up others, while giving them an outlet if they must go mucking around. It might not even be a bad idea in a setting like Jim's lab to chmod -w this file after it is set up correctly, as an additional deterrent against doing something stupid.

  • Christopher Stawarz

    Christopher Stawarz May 18th, 2010 @ 12:51 PM

    • State changed from “open” to “resolved”

    These changes are now in place. Data files are now stored in ~/Documents/MWorks/Data. When loading the config file, the core now looks first in ~/Library/Application Support/MWorks/Configuration. If it finds a config file there, it uses it. Otherwise, it looks in /Library/Application Support/MWorks/Configuration. Finally, experiment files are now cached in the directory returned by NSTemporaryDirectory, in the subdirectory MWorks/Experiment Cache. To get the path the current user's temp dir (e.g. for debugging), you can use the following command:

    /usr/bin/python -c 'from Foundation import * ; print NSTemporaryDirectory()'
    

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

The core framework and supporting libraries for the MWorks Suite

People watching this ticket

Referenced by

Pages