diff options
-rw-r--r-- | devdoc/viewer_index_v2.1.md | 167 |
1 files changed, 167 insertions, 0 deletions
diff --git a/devdoc/viewer_index_v2.1.md b/devdoc/viewer_index_v2.1.md new file mode 100644 index 0000000..0595a55 --- /dev/null +++ b/devdoc/viewer_index_v2.1.md | |||
@@ -0,0 +1,167 @@ | |||
1 | --- | ||
2 | title: "Viewer: index v2.1" | ||
3 | author: pacien | ||
4 | date: 2022-10-25 (v3) | ||
5 | --- | ||
6 | |||
7 | # Abstract | ||
8 | |||
9 | This is a short documentation of the viewer's internal index dating from the | ||
10 | first version of this software and still currently in use in version 2.1. | ||
11 | |||
12 | This index was written without proper specifications or documentation, and has | ||
13 | evolved a lot since then. | ||
14 | |||
15 | This document is an attempt at describing how this index is currently being | ||
16 | generated and used, before further work can be done on some proper | ||
17 | specification to clarify the problematic behaviours. | ||
18 | |||
19 | |||
20 | # Document version history | ||
21 | |||
22 | 1. 2022-10-15 by pacien: call notes | ||
23 | 2. 2022-10-22 by pacien: rewritten | ||
24 | 3. 2022-10-25 by pacien: include feedbacks from zeroinformatique | ||
25 | |||
26 | |||
27 | # Description of the current implementation | ||
28 | |||
29 | ## Generating the index(es) from the item tree | ||
30 | |||
31 | The gallery item index (`index.json`) is loaded by the `galleryStore` and made | ||
32 | available as the `galleryIndex` attribute. | ||
33 | |||
34 | Two indexes are derived from that file through the `indexFactory` service: | ||
35 | |||
36 | - `tagsIndex`: | ||
37 | - Maps each tag "part" (component) to items tagged with it. | ||
38 | - Maps tags to their immediately preceding disambiguating component as child. | ||
39 | - Includes a normalised version of the tag for searching. | ||
40 | - Stores the `childPart`, used to generate tag categories indexes. | ||
41 | - (Stores the left-most `rootPart` component. This is not used anywhere). | ||
42 | |||
43 | - `tagsCategories`: | ||
44 | - Same as `tagsIndex`, but partitioned by tag category prefixes as defined in | ||
45 | the viewer configuration. | ||
46 | |||
47 | |||
48 | ## Search query input auto-completion "suggestions" | ||
49 | |||
50 | The `TagInput` component suggests tags as the user types in that input field. | ||
51 | |||
52 | This is provided by the `indexFactory.searchTags` function in fuzzy | ||
53 | (non-strict) mode, which only uses the global tag index. | ||
54 | |||
55 | The fuzzy search is implemented as a normalised (lowercase) infix word lookup, | ||
56 | including the disambiguating tag parts. | ||
57 | |||
58 | The auto-completion suggestions are independent of the current directory. | ||
59 | Suggestions yielding no result (incompatible with the current search query) are | ||
60 | not excluded either. The current implementation makes the choice of suggesting | ||
61 | everything because the current index does not allow finer filtering, and | ||
62 | because walking the whole item tree may lead to performance issues. | ||
63 | |||
64 | |||
65 | ## Item search | ||
66 | |||
67 | The search query is stored in the URL query. This allows a search to be shared | ||
68 | by copying the current URL. | ||
69 | |||
70 | This URL search query is updated by `LayoutLeft.vue` to match modifications | ||
71 | made through the tag input or related filters "propositions". This component | ||
72 | also updates the store to match the URL query through `galleryStore.search`. | ||
73 | |||
74 | A search query consists of three sets of tags: an intersection list, a forced | ||
75 | inclusion (union) list, and a forced exclusion list. The last two are denoted | ||
76 | with a `+` and `-` modifier prefix before the tag name. The order of the terms | ||
77 | does not matter. | ||
78 | |||
79 | The result computed in `indexSearch.indexSearch` is given by | ||
80 | `(⋂(intersection) ∪ ⋃(forced inclusion)) ∖ ⋃(forced exclusion)`. | ||
81 | |||
82 | The string representation of a query is parsed in `indexFactory`. It is | ||
83 | serialised by taking the `filter.display` property of filters, in | ||
84 | `LayouLeft.vue` for being displayed in the tag input and in the URL. | ||
85 | |||
86 | |||
87 | ## Related filters "propositions" | ||
88 | |||
89 | The left pane of the user interface lists related filters "propositions", | ||
90 | related to the current search results or directory being viewed. | ||
91 | |||
92 | Tags in that pane are grouped according to the `tagCategories` gallery | ||
93 | configuration key. (This is currently buggy: some tags can appear in the wrong | ||
94 | category under some circumstances). | ||
95 | |||
96 | The related tags are filtered with respect to the current search query or | ||
97 | directory: only tags that are present on the listed items are shown. | ||
98 | |||
99 | Each "proposed" tag has an occurrence count of the items having that tag in the | ||
100 | whole gallery. (This is inconsistent with the locality of the filter). | ||
101 | |||
102 | This is computed using a full gallery search through the `galleryStore` using | ||
103 | `indexFactory.searchTags` in strict (non-fuzzy) mode. | ||
104 | |||
105 | |||
106 | # Identified issues and proposals | ||
107 | |||
108 | ## Issues affecting the end users | ||
109 | |||
110 | - Tags categories and disambiguation aren't properly defined: | ||
111 | - It is not clear whether intermediate tag components should be treated as | ||
112 | tags and suggested at all. (They currently are). | ||
113 | |||
114 | - Tags with indirect disambiguations are not handled correctly: | ||
115 | - Example in `a:b:c`: | ||
116 | - `b` is a child of `a`, `c` is a child of `b`. | ||
117 | - But `c` is not registered as a child of `a` in `tagsIndex`. | ||
118 | |||
119 | - Homonymous disambiguated tags are not handled in separate categories. | ||
120 | - Example with `a` and `b:a`: | ||
121 | - `a` seems to be shown under category `b`. | ||
122 | - This seems to be the cause of tags being displayed in the wrong category in | ||
123 | the suggestion pane. | ||
124 | |||
125 | - The tag input's auto-completion suggests impossible intersections: | ||
126 | - The fuzzy (non-strict) search does not work the same way as the suggestions | ||
127 | panel, which restricts the suggestions. | ||
128 | - This might however be problematic for forced inclusions (union) tags | ||
129 | which are still meaningful. | ||
130 | - They could still be listed but greyed for example. | ||
131 | |||
132 | - The tag occurrence counts in the related tags "propositions" pane is | ||
133 | misleading: | ||
134 | - This view suggests only the tags for the current search results | ||
135 | (descendants of the current directory and matching the current search query | ||
136 | if any), | ||
137 | - But the occurrence count for each tag is global (on the whole gallery | ||
138 | instead of the current search results). | ||
139 | |||
140 | |||
141 | ## Issues affecting only the developers | ||
142 | |||
143 | - Ambiguous terminology: | ||
144 | - For example "index" vs "index.json", or "tag suggestions" vs | ||
145 | "tag propositions" vs "tag completion". | ||
146 | - This confusion is reflected in the component naming and coupling… | ||
147 | - A glossary and would help. | ||
148 | - Refactoring and renaming the modules would help. | ||
149 | |||
150 | - Tight coupling of the tag-related and index operations: | ||
151 | - It goes all over the place. | ||
152 | - Some concerns should and can clearly be separated for example: | ||
153 | - For example query parsing, compiling and actual run on the item tree. | ||
154 | - The new modules should make use of composition with the rest of the | ||
155 | components. | ||
156 | |||
157 | - Lack of unit tests: | ||
158 | - Coupling is preventing easy unit testing. | ||
159 | - Once the concerns are separated: | ||
160 | - We'll have clear expected outputs with respect to some input. | ||
161 | - It should be easier to do unit testing: | ||
162 | - (perhaps through randomised property testing). | ||
163 | |||
164 | - Minor: relatively verbose and intertwined imperative code: | ||
165 | - The query parsing and recursive tree operations would probably be more | ||
166 | elegant in PureScript than Javascript/Typescript. | ||
167 | - Same with unit and property tests. | ||