Merge pull request #333 from ldgallery/p_devdoc_viewer_index_2.1

devdoc: describe the viewer's index as of v2.1

1 files changed, 167 insertions, 0 deletions

diff --git a/devdoc/viewer_index_v2.1.md b/devdoc/viewer_index_v2.1.md
new file mode 100644
index 0000000..0595a55
--- /dev/null
+++ b/devdoc/viewer_index_v2.1.md
@@ -0,0 +1,167 @@
+---
+title: "Viewer: index v2.1"
+author: pacien
+date: 2022-10-25 (v3)
+---
+
+# Abstract
+
+This is a short documentation of the viewer's internal index dating from the
+first version of this software and still currently in use in version 2.1.
+
+This index was written without proper specifications or documentation, and has
+evolved a lot since then.
+
+This document is an attempt at describing how this index is currently being
+generated and used, before further work can be done on some proper
+specification to clarify the problematic behaviours.
+
+
+# Document version history
+
+1. 2022-10-15 by pacien: call notes
+2. 2022-10-22 by pacien: rewritten
+3. 2022-10-25 by pacien: include feedbacks from zeroinformatique
+
+
+# Description of the current implementation
+
+## Generating the index(es) from the item tree
+
+The gallery item index (`index.json`) is loaded by the `galleryStore` and made
+available as the `galleryIndex` attribute.
+
+Two indexes are derived from that file through the `indexFactory` service:
+
+- `tagsIndex`:
+  - Maps each tag "part" (component) to items tagged with it.
+  - Maps tags to their immediately preceding disambiguating component as child.
+  - Includes a normalised version of the tag for searching.
+  - Stores the `childPart`, used to generate tag categories indexes.
+  - (Stores the left-most `rootPart` component. This is not used anywhere).
+
+- `tagsCategories`:
+  - Same as `tagsIndex`, but partitioned by tag category prefixes as defined in
+    the viewer configuration.
+
+
+## Search query input auto-completion "suggestions"
+
+The `TagInput` component suggests tags as the user types in that input field.
+
+This is provided by the `indexFactory.searchTags` function in fuzzy
+(non-strict) mode, which only uses the global tag index.
+
+The fuzzy search is implemented as a normalised (lowercase) infix word lookup,
+including the disambiguating tag parts.
+
+The auto-completion suggestions are independent of the current directory.
+Suggestions yielding no result (incompatible with the current search query) are
+not excluded either. The current implementation makes the choice of suggesting
+everything because the current index does not allow finer filtering, and
+because walking the whole item tree may lead to performance issues.
+
+
+## Item search
+
+The search query is stored in the URL query. This allows a search to be shared
+by copying the current URL.
+
+This URL search query is updated by `LayoutLeft.vue` to match modifications
+made through the tag input or related filters "propositions". This component
+also updates the store to match the URL query through `galleryStore.search`.
+
+A search query consists of three sets of tags: an intersection list, a forced
+inclusion (union) list, and a forced exclusion list. The last two are denoted
+with a `+` and `-` modifier prefix before the tag name. The order of the terms
+does not matter.
+
+The result computed in `indexSearch.indexSearch` is given by
+`(⋂(intersection) ∪ ⋃(forced inclusion)) ∖ ⋃(forced exclusion)`.
+
+The string representation of a query is parsed in `indexFactory`. It is
+serialised by taking the `filter.display` property of filters, in
+`LayouLeft.vue` for being displayed in the tag input and in the URL.
+
+
+## Related filters "propositions"
+
+The left pane of the user interface lists related filters "propositions",
+related to the current search results or directory being viewed.
+
+Tags in that pane are grouped according to the `tagCategories` gallery
+configuration key. (This is currently buggy: some tags can appear in the wrong
+category under some circumstances).
+
+The related tags are filtered with respect to the current search query or
+directory: only tags that are present on the listed items are shown.
+
+Each "proposed" tag has an occurrence count of the items having that tag in the
+whole gallery. (This is inconsistent with the locality of the filter).
+
+This is computed using a full gallery search through the `galleryStore` using
+`indexFactory.searchTags` in strict (non-fuzzy) mode.
+
+
+# Identified issues and proposals
+
+## Issues affecting the end users
+
+- Tags categories and disambiguation aren't properly defined:
+  - It is not clear whether intermediate tag components should be treated as
+    tags and suggested at all. (They currently are).
+
+- Tags with indirect disambiguations are not handled correctly:
+  - Example in `a:b:c`:
+    - `b` is a child of `a`, `c` is a child of `b`.
+    - But `c` is not registered as a child of `a` in `tagsIndex`.
+
+- Homonymous disambiguated tags are not handled in separate categories.
+  - Example with `a` and `b:a`:
+    - `a` seems to be shown under category `b`.
+  - This seems to be the cause of tags being displayed in the wrong category in
+    the suggestion pane.
+
+- The tag input's auto-completion suggests impossible intersections:
+  - The fuzzy (non-strict) search does not work the same way as the suggestions
+    panel, which restricts the suggestions.
+    - This might however be problematic for forced inclusions (union) tags
+      which are still meaningful.
+    - They could still be listed but greyed for example.
+
+- The tag occurrence counts in the related tags "propositions" pane is
+  misleading:
+  - This view suggests only the tags for the current search results
+    (descendants of the current directory and matching the current search query
+    if any),
+  - But the occurrence count for each tag is global (on the whole gallery
+    instead of the current search results).
+
+
+## Issues affecting only the developers
+
+- Ambiguous terminology:
+  - For example "index" vs "index.json", or "tag suggestions" vs
+    "tag propositions" vs "tag completion".
+  - This confusion is reflected in the component naming and coupling…
+  - A glossary and would help.
+  - Refactoring and renaming the modules would help.
+
+- Tight coupling of the tag-related and index operations:
+  - It goes all over the place.
+  - Some concerns should and can clearly be separated for example:
+    - For example query parsing, compiling and actual run on the item tree.
+  - The new modules should make use of composition with the rest of the
+    components.
+
+- Lack of unit tests:
+  - Coupling is preventing easy unit testing.
+  - Once the concerns are separated:
+    - We'll have clear expected outputs with respect to some input.
+    - It should be easier to do unit testing:
+      - (perhaps through randomised property testing).
+
+- Minor: relatively verbose and intertwined imperative code:
+  - The query parsing and recursive tree operations would probably be more
+    elegant in PureScript than Javascript/Typescript.
+  - Same with unit and property tests.