[Dev Notes] Word Segmentation and Smear-to-Select Feature

Preface

Recently, I've been working on an emoji search feature. Since it's a search, emojis need some recognizable attributes—so assigning each emoji a name field makes sense. But a user might have thousands of emojis—naming them one by one would be a nightmare. That’s why we need OCR for emojis. However, OCR isn’t always reliable, so we also need to allow users to manually adjust the names afterward.

But tapping to open an input box and then editing text is still too much friction. Inspired by Luo Yonghao’s “Big Bang” feature, selecting segmented text by smearing (dragging across it) offers a much better interaction experience.

Target Effect

Let’s look at the effect first:

Each segmented word is placed inside a box. Selected words are highlighted. Tapping or swiping lets you select/deselect them, with support for select all / deselect all.

Implementation Details

Refer to my previous post where I explain the implementation and logic behind swipe-to-select:

Previous Article Link (Huawei Developer Blog)

We’ll modify that implementation for our needs.

The biggest challenge here is dealing with the irregular layout of word blocks from left to right. For those familiar with web development, Flex layout’s flex-wrap property immediately comes to mind!

This official flex-wrap demo is exactly what we want.

Once the tech stack is set, we begin coding.

---

First, let’s extract and refine the swipe selection logic from the previous article into a basic library:

@ObservedV2
class GridSlideSelectInfo {
  ...
}
@ObservedV2
export class SlideToSelectGridViewModel {
  ...
}

Next, we create a component framework. The parameters needed: words to hold all the segments, and selectedIndex to track which words are selected.

@ComponentV2
export struct WordDividerContent {
  @Param @Require words: string[]
  @Param @Require selectedIndex: Set<number>

  build() {}
}

Quick note on how to do the word segmentation: if you have your own server, just request results from there. Otherwise, check out HarmonyOS AI’s textProcessing API:
HarmonyOS Word Segmentation Docs

textProcessing.getWordSegment(text)
  .then((res) => {
    this.aiDividedWords = res.map(val => val.word).filter(val => !val.includes('\n'))
  })

This gives you aiDividedWords: string[].

Now, onto the UI.

First, use Flex layout with flex-wrap to handle word block layout:

build() {
  Column() {
    Scroll() {
      Flex({
        direction: FlexDirection.Row,
        wrap: FlexWrap.Wrap,
        ...
      }) {
        ForEach(this.words, (val: string, index) => {
          Text(val).selectedStyle(<selection logic>)
        })
      }
    }
  }
}

Custom style based on whether a word is selected:

@Extend(Text) function selectedStyle(selected: boolean) {
  .backgroundColor(selected ? ThemeManager.getBrand() : $r('sys.color.background_tertiary'))
  ...
}

Add logic for click-to-select:

Text(val)
  .selectedStyle(this.judgeSelected(index))
  .onClick(() => {
    ...
  })

Then we add area tracking and pan gesture handling:

@Local slideVM: SlideToSelectGridViewModel = new SlideToSelectGridViewModel()
private wordArea: Area[] = []

aboutToAppear(): void {
  this.wordArea = new Array(this.words.length)
}

Add onAreaChange to each word block to track its area:

Text(val)
  .onAreaChange((_o, area) => { this.wordArea[index] = area })

Track the full component’s area and bind gestures:

.onAreaChange((_, nv) => {
  this.slideVM.pageArea = nv
})
.gesture(PanGesture().onActionStart(...).onActionUpdate(...).onActionEnd(...))

Now implement swipe-to-select logic, including a finder to locate which block a touch point is over:

private finder = (vx: number, vy: number) => {
  return this.wordArea.findIndex(val => this.slideVM.checkIsInArea(vx, vy, val))
}

Update judgeSelected() to handle swipe states:

judgeSelected(index: number) {
  ...
}

Handle gesture callbacks:

this.slideVM.onSlideActionStart(...)
this.slideVM.onSlideActionMove(...)
this.slideVM.onSlideActionEnd(...)

Gesture logic is encapsulated in the view model for cleanliness:

class SlideToSelectGridViewModel {
  public onSlideActionStart = ...
  public onSlideActionMove = ...
  public onSlideActionEnd = ...
}

Finally, add two buttons for “Select All” and “Deselect All”:

Row() {
  ActionButton({ title: 'Deselect All', clickAction: () => { ... } })
  ActionButton({ title: 'Select All', clickAction: () => { ... } })
}

And we’re done!