Jun 12, 2022
In General Discussion
This part is the part with the most pitfalls and the most problems, which requires repeated communication between PM and labelers. , repeated iterations, and finally can deliver executive list better labeled data for training the model. Two kinds of labeling processes are given here, and which one to use needs to be decided according to its own business characteristics executive list and personnel configuration. The first one is a single layer, where senior annotators are responsible for task issuance, sampling inspection, and personnel training. This is suitable for executive list senior annotators with excellent quality and strong ability. The advantage is that there are fewer intermediate processes, and information transmission is not prone to deviations. Time is also long. Recommendation Strategy Product Manager: Two or Three Things to Build a Labeling System Single layer annotation process The second is a double-layered layer, adding a layer of intermediate between the advanced and primary annotations, responsible for labeling quality inspection and determining executive list some uncertain cases. This advantage is that each person is responsible for relatively few things, and it is not easy to make mistakes, and Responsibility to people, relatively low requirements for personal ability. Recommendation Strategy Product Manager: Two or Three Things to Build a Labeling System executive list Double-layer labeling process In the double-layer, the specific responsibilities of each party are as follows (T in [T] represents the current day, and T+1 represent. Responsible for data extraction [T] Align cases that senior annotators are not sure about[T+1] Update and Amendment of Evaluation Criteria Senior Annotator: Responsible for the disassembly and distribution of labeling tasks [T] Align the cases that the executive list intermediate annotators are not sure about, and give the classification based on their own judgment. If the individual cannot be determined, then align with the PM [T+1: align before get off work] Standards executive list update, communication of alignment standards and personnel training [T+1] Intermediate annotators: End every day when the junior annotators are not sure about the case, give a classification based on their own judgment, and can not be determined and then align.