The Return of the Classic Pirates contest was a joint venture between Classic-Pirates.com and the LEGO Pirates Forum on Eurobricks.
The contest was divided into two building categories; the Main Building Category for larger entries, and a Mini Set Building Category for limited to 50 pieces, while a third category, the Creative Critic catered towards literary based entrants.
What Is the Creative Critic?
The Creative Critic is an initiative which serves two purposes:
- To grant accessibility to a wider audience.
Constructing MOCs is but one facet of AFOL interest. LEGO Pirate fans may possess vast love for the LEGO Pirates theme, but lack the space, bricks or skill to submit a building entry. Thereby, offering a category which allows participation in written form, affords opportunities to entrants who possess an entirely different skill set.
- Simulate engagement in building entry topics.
Building entrants were required to create a new forum topic (thread) for each creation they submitted, however this alone does not guarantee forum members response. The Creative Critic was envisaged as further incentive to provide direct feedback upon contest entries while stimulating the discussion surrounding them.
Like the building categories, a prize was offered to the Creative Critic deemed most successful.
The Creative Critic winner received a 10320 Eldorado Fortress
Given the sizeable commitment required to win, the prize had to be substantial to justify the effort, and thus the newly released 10320 Eldorado Fortress was offered.
Did the Creative Critic Succeed?
In terms of fulfilling its core objectives; To grant accessibility to a wider audience and stimulate engagement in building entry topics the Creative Critic was highly effective.
Typically, Eurobricks forum-based contests with dedicated entry topics receive less than 5 responses per topic, not so for the Return of the Classic Pirates contest!
The majority of earlier entry topics received a high level of engagement with some receiving more than 50 replies. As to be expected, this level of activity waned towards the conclusion of the contest, attributed to finite opportunity to engage prior to the deadline and writer’s fatigue.
Criticism
Scrutiny was cast towards the level of reporting and analysis employed to determined the Creative Critic winner.
Previous Creative Critic initiatives lacked any form of scoring or reporting whatsoever, and the winners were simply announced without subsequent conjecture regarding their selection – see Pillage the Village Contest, Pillage the Village II and Pirates of the Caribbean Contest. Therefore, the requirement to comprehensively report was not anticipated for the The Return of the Classic Pirates contest.
Nonetheless, an effort to gauge each Creative Critics performance was presented via a leaderboard comprised of a graphic visualization with the inclusion of feedback analyzing each registrants’ activity.
However, due to insufficient management capacity, the Creative Critic Leaderboard was not updated as frequently as intended, nor was the final reporting available to coincide with the winner announcement. Consequentially, this prompted some to refute the integrity of the winner selection.
While the concerns raised were considered valid, generally AFOL communities and fan websites lack the personnel and resources to provide the same level of reporting offered by traditional leagues and clubs.
How Was the Creative Critic Winner Determined?
The Creative Critic was not a sweepstakes in which a winner was selected at random. Each critic was assigned a score between 1 and 100 based determined by the quality and frequency of their contributions to contest entry and administration topics.
Various criteria was employed to determine this score.
Written Assessment Criteria
These criteria assessed the critics’ writing abilities:
- The helpfulness of each critics’ feedback.
- Whether the building entrant considered or actioned feedback received by critics.
- Whether the critics’ engagement stimulated the discussion surrounding the entry.
Activity Assessment Criteria
These criteria assessed the level of participation in relation to other critics:
- When the critic commenced and/or ceased participation.
- Whether the critics’ feedback encompassed a wide variety of entries spanning the entire duration of the contest, or just entries submitted during a certain time frame.
- Whether the critic followed up on their initial feedback and the frequency of their response when directly engaged
- The length and comprehensiveness of the critics’ responses
- Whether the critic also participated in contest administration topics or just entry topics.
- Whether the critic took creative initiatives beyond the initial expectations.
- Whether the critic adhered to the Creative Critic terms and conditions.
Scoring
Initial Preliminary Score
Scoring was not conducted during the first 5 weeks of the contest due absence of the official, and the first score was determined for each critic on 10 August, 2023.
Creative Critics who had not yet participated received a score of zero, while a preliminary score was calculated for active critics by reviewing their engagement throughout the previous month based on the assessment criteria.
Their score was then used to plot a bar chart to visualise their ranking:
As a further measure to clarify their performance, active critics were provided feedback highlighting their most prominent strengths and weaknesses, along with brief direction for improving their score during future assessments.
Updated Preliminary Score
21 August, 2023 saw an update to the Creative Critic Leaderboard.
To streamline reporting critics who had formally withdrawn or yet to participate were not included, while active critic scores were adjusted accordingly:
- If a critics’ engagement had improved their score was increased proportionally to the quality and frequency of their overall contribution.
- However, if their engagement had discontinued their score was reduced proportionally to the overall level of critic participation.
- Or if the quality of their engagement reduced, their score was reduced proportionally to the quality of their overall participation.
Each critics’ feedback was updated in accordance with their activity since the initial scoring. Observable differences in their participation were noted, whilst feedback remained unamended for critics whose activity had ceased.
A LEGO Pirates Forum screenshot of the updated feedback provided to Creative Critics
Unpublished Update
A third Creative Critic Leaderboard update was intended for 16 September, 2023 but not finalised and published due to time constraints.
Utilising notes and scores recorded between the 21 August and 16 September, that update became available 21 October, 2023.
Each critics’ feedback was also elaborated upon to reflect their updated score, taking into account their activity during the last five days of the entry submission period and the subsequent three weeks of voting.
Feedback From the Building Entrants
In the interest of acquiring a broader perspective of critic activity, particularly from building category entrants directly affected by Creative Critic engagement, a topic was launched to obtain feedback. The purpose of this topic was not intended for outright voting, rather an avenue that allowed further insight into critic engagement which may influence the final decision.
Unfortunately participation was minimal and yield insufficient data to form any definitive conclusions beyond what scoring had already indicated. Had building entrant activity been more substantial, this discussion would have held greater influence over the final result.
Determining the Winner
To further streamline the evaluation process, critics who had not been active since the 21 August update were removed from the leaderboard as their scores lacked sway over the active critics final scores.
The final scores at 23 September, 2023
The stalwarts of the Creative Critic; Horation, Rogue Redcoat, MstrOfPppts, iragm and Mazin were all at one point, contenders for the winning position, attributed to:
- their high level of participation throughout the majority of the contest,
- coverage of a wide variety of entries submitted through the lifespan of the contest,
- while maintaining an exemplary level of quality,
- with consideration for the feedback they were provided.
- and their willingness to participate beyond the initial requirements. I.e. also participating in contest administrative discussions.
Further evaluation postulated which specific contributions could place a critic ahead of their competition.
Process of Elimination
Horation
Horation was a strong contender for the winning position during the first two-thirds of the contest but formally withdrew 11 August due to personal reasons.
thewatchman
While also demonstrating a high level of participation through to the winner announcement, he did not fully action the feedback provided which hampered the reward of generous score increases.
Thewatchman’s preliminary score of 55 peaked at 70 at the conclusion of the contest.
Yoggington
Yoggington who’s participation experienced latency during mid contest, returned strong towards the end of August. His provision of voting rationales and a precursory voting tally for building entries (which would later act as the basis for the final vote tally) catapulted his score from around 40 at mid contest, to 85 at the conclusion of the contest.
However, Yoggington’s overall critique during the final phase of the contest was not to the same extend as other critics.
Duranzo33
Duranzo33 was another critic who experienced a hiatus but reemerged towards the end of August with consistent, high quality feedback. Initially, Duranzo’s critique was not as comprehensive as other high performing critics, but this improved substantially within a couple of days and his final contributions were on par with other finalists.
He also produced insightful video reviews for building entries and his voting rationales. While the Creative Critic is intended to be a literacy-based contest, extra points were awarded for taking initiative. Nonetheless, merit for producing video must never undermine the imperative of the Creative Critic’s primary objective.
Further considerations which governed Duranzo’s final score included:
- the length of hiatus, which was substantially longer than Yogginton’s hiatus.
- he did not participate in administrative contest topics prior to the winner announcement
Even so, a generous score increase was awarded, elevating his score from around 30 at the commencement of his hiatus, to 76 at the conclusion of the contest.
Rogue Redcoat
Rogue Redcoat remained firmly within the ranks of the top 5 critics, particularly because his feedback stimulated discussion in entry topics. Has also contributed suggestions to administrative topics and demonstrated effective proofreading skills.
His biggest shortcomings, however:
- his follow-up communication was not to the same caliber as other high ranking critics on the basis it was considerably briefer and less informative.
- he failed to provide voting rationales.
His preliminary score of 75 increased to 90 by the conclusion of the contest.
Top Three Critics
It became increasingly challenging to select one critic for the prize winning position, with first preference alternating between MstrOfPppts, Iragm and Mazin on a near daily basis. Had first, second and third prizes been available for the Creative Critic, these three participants would have all been awarded.
A commonality they shared was expanding upon their initial feedback in elaborate detail, rather than falling into immediate complacency. Less successful critics tended to focus their efforts in their initial post while their subsequent posts were perfunctory, seemingly made as a cursory obligation.
So what hoisted Mazin to the top?
MstrOfPppts and Iragm both scored highly during the preliminary rankings, at 85 and 90 respectively, while Mazin trailed generously at 70 due to neglect of entries submitted during the first month of the contest.
However, by the 21 August update he’d more than compensated for this deficiency and was contributing at a comparable level of engagement to that of MstrOfPppts and Iragm.
Overall, Iragm and MstrOfPppts feedback was more direct and technically orientated, while Mazin’s feedback was more personable, as though he were guiding, rather than informing. His affable comport slightly nudged his score ahead of the other contenders.
This decision was fueled by subjectivity and had more judging officials been available, or more building entrants provided endorsements during the Let’s determine the Creative Critic WINNERS discussion, Iragm could well have been nominated as the Creative Critic winner. Some may argue, Marooned Marin’s endorsement alone should have secured Iragam’s victory.
Nonetheless, all three performed spectacularly with minuscule difference in their final scores; MstrOfPppts at 97, Iragam at 98 and Mazin at 100.
Isn’t this Report a Tad Overdue?
Aye, almost 6 weeks have flown by since the contest winner announcement. If yer would like a long winded exposition behind the delay, cue the violins and sail over to the LEGO Pirates Forum and indulge upon the Formalities of the Return of the Classic Pirates contest. There be the precise details.
The more concise reason is; insufficient personnel to manage the contest at the capacity required to provide comprehensive reporting.
Challenges of Assessing the Creative Critic
It’s a monumental task to track the output of each critic in conjunction with other content management responsibilities.
Had this task been assigned to dedicated officials, whose sole responsibility was to review and report upon critic performance, the reporting could have been more thorough.
Multiple officials could have also reduced or eliminated subjectivity from the judging decisions.
What Can Be Done?
We hope to rectify this shortcoming for future contests – but we need YOUR help so we can have improved reporting!
If you wish to lend a hand to the Classic Pirates cause, reach out. Drop by the LEGO Pirates Forum and introduce yerself, or complete our Contact Form.
Whether yer have 5 hours or 5 minutes to spare… for blogging, graphic design, image editing, illustration, creative writing, photography, data analysis, Creative Critic judging, contest officiating… we can allocate a role to suit yer strengths and schedule. Just tell us what yer passions be!
Special thanks to Yperio Bricks, Rogue Redcoat and Captain Pirate Man who’ve already pledged their support in the wake of the recent contest.
What Do Yer Think?
Do you think the right Creative Critic was chosen as winner? Or do you think another critic should have taken the prize? If so, who?
How could the Creative Critic winner have been determined differently? Or do you think this level of scoring and reporting is unnecessary?
Stop by the LEGO Pirates Forum to share yer thoughts on the latest Creative Critic.
