Use Cases --------- Monitor: [M1] /dir/*/*.log ; tag="monitor1" [M2] /dir/abc/a.* ; tag="monitor2" [M3] /dir/*/a/*.txt ; tag="monitor3" [M4] /di?/*/b/*.txt ; tag="monitor4" - if a.log is created, we have kind of a conflict: create two monitors then? (sounds best so far) - /dir/abc may not initially exist, but later on created -> same problem for any path depth (polling? multiple inotify? fail?) Let's assume /dir/abc exist, but nothing else under /dir Setup initial inotify: /dir [M1] /dir [M3] /dir/abc [M1] /dir/abc [M2] /dir/abc [M3] /dir/abc [M4] Rule? The parent of each component must be monitored / [M4] -- really??? Means we need to monitor whole FS tree! /dir [M2] - /dir/abc may not yet exists but may be created later on Q: can we combine /dir/abc? A: well, we need to know what shall be monitored. e.g. /dir/abc/a.logfile is created watch /dir/abc must be evaluated does not match [M1], but [M2]! Event for /dir/abc is generated need to check [M1], [M2] /dir/def is created watch /dir must be evaluated --> [M1], [M2], [M3], [M4] must be checked matches [M1],[M3],[M4] create new watch /dir/def [M1, M3, M4] - this watch should not exist if all is right [-> assert()] /dir/def/new.log is created watch /dir/def must be evaluated --> [M1, M3, M4] create new file watch /dir/def/new.log [M1] /dir/abc/a.log is created watch /dir/abc must be evaluated --> [M1, M2, M3, M4] create new file watch /dir/abc/a.log [M1, M2] At each level, I must know which child components to watch out for, and which monitor/listener they belong to. Comparison must be done on immediate-child level. Data Structures -------------- Monitors ........ - path needs to be split into individial components --> array File Watches ............ - contains active file state - one file watch per file per monitor (/dir/abc/a.log sample with two active file entries) - multiple entries are not properly supported by inotify (and possibly FEN) --> need to multiplex ourselfes based on single notifcation Dir Watches ........... - contains active directory state - used to discover new to-be-monitored files Problems -------- - we need to find, for each level, the proper "next component" that we need to check against (fnmatch on component?) - this must be done om a per-monitor basis - one entry per monitored file, but a list of monitors (and component names to check against) --> how do we efficiently describe the monitor so that we can do an O(1) name comparison? --> we can NOT assume that the full path matches, just component-by-component! --> we may NOT get notifications if path level are rapidly created, as we may not have setup the watch sufficiently fast - so we need to poll the full tree in any case - state files: different state if the same file is monitored by more than one monitor --> or should we flag this as an error case? Poll vs. Watch -------------- Polling is always needed (due to watch racieness). So it probably is best to have some generic poll code, and use watches just to reduce the amount of polling. That also means that poll mode can do exactly the same the notification mode can do! Procedures ---------- e.g. /dir/aaa is created this will be reported on /dir watch, which is watched by all 4 monitors foreach monitor: check if next-level component matches - this is the case for [M1, M2, M3] if so: create new/update existing entry (existing for all but first!) arm watch for this entry (if newly created) do a full poll of that entry and its subentries AS FAR AS THE MONITOR IN QUESTION is affected: M1 will scan subdir for "*.log" M2 will scan for "abc", if it exists scan for "a.*" M3 will scan for "a", if it exists scan for "*.txt" --> rest of the path components need to be scanned, files processed for each monitor this is a recursive function The above-mentioned poll process is fundamental, and can occur on each level of the file system/monitor definitions. It needs to work both on active monitors as well as the configured path components. Does ist make sense to have them in a single structure? Is a tree more useful than an dirs table? now the opposite: e.g. /dir/aaa is deleted this will be reported on /dir watch, which is watched by all 4 monitors check if "aaa" exists as active(dynamic) entry if so: check for sub-entries, initiate delete events if any exists (TODO: is MOVE here a problem?) delete "aaa" entry Note: we MUST NOT remove entries that are statically configured (e.g. /dir in our samples, even if it is deleted - it may re-appear)