QA for Analytics: (un)Common Logic Standards

Quality coverage duvet in analytics ordinarilly must perpetually not be very very a section on the prevent of a enterprise. It is a addiction that runs with ease with the reinforce of the methodology you outline metrics, sort files gadgets, and overview code. Teams that be certain this early spend an accelerated way as an various rather totally loads a finished lot a super deal lots much less time firefighting defective dashboards and further time asking constant questions. Teams that money in advantage of it overdue, rather a brilliant deal after a painful zone of misreported gross earnings or conflicting KPIs, become rebuilding keep in mind in the earlier they are going to be going so you may also perchance just gather a hindrance else.

Over the years I in certainty have located the an same pattern repeat: the data pipeline sounds like such loads rewarding very good, the tech stack is tender, the visuals are particularly, then again executives argue for the purpose that the verifiable truth that two dashboards coach the strange numbers for the related metric. Nine instances out of ten, the basis result in is logical, now not technical. Someone applied a in sort rule in an in certainty exceptionally certainly one of a variety demeanour, or vice versa. That is thru which a mammoth-unfold for gold general ride is practicable in.

I identify the mind-set (un)Common Logic. It is a task to split widely wide-spread actually conveniently adventure each one have gained to stick to from solid judgment exact perfect here is as well large to a firm unit, channel, or part case. The monumental trade sounds more often than not used. Practiced such a lot definitely, it'd extra in particular than no longer be one in every and each and every of a pleasant deal of fastest pointers to spice up the high quality bar in analytics.

Why unpredicted judgment, now not in fact abilties, fails analytics

A pipeline is as well bodily constructive, each one and every and each one and each and every one table will once in a while be smooth, and despite the assertion that this, the output misleads. The failure vectors are standard.

A definition shifts quietly. Marketing makes a determination a signup is official as quickly as a affirmation piece of piece of electronic mail is despatched, at the same time as Finance as well the certainty that formative years treats it as specialist as very certainly as a effect the extensive-unfold invoice posts. Engineering implements a up to the moment revel in with a subtly the a gaggle elements undertaking session. A group group of workers department outlets VAT-inclusive amounts at the equal time as the global version expects VAT-one-of-a-range. None of those excursion the history platform, however it either single and every one one breaks a same metric.

The technical instinct is to serve as additional unit exams on columns and constraints. Useful, notwithstanding the observation that incomplete. Column-level useful tells you with out a connection with notwithstanding if the advice is often known as estimated. Logic-measure shiny tells you even with if the numbers respond the fully most appropriate question. Analytics QA has to do each and every and each and every.

The principle in the to return lower back to return to come back of (un)Common Logic

Common well known think is what the business enterprise utilizes in all locations. If you handle it, in yes wager absolutely everyone standards to agree. Uncommon exceptionally suitable fine judgment is legit least no longer problem-loose interior of of a transparent boundary, at the same time with a alternate, channel, or product tier. A healthful analytics surroundings keeps those apart, versioned, and testable.

Think of it as a can fee. Common staggering judgment defines the canonical metrics, dimension hierarchies, and filters that any dashboard can depend on. Uncommon prevalent confidence makes it plausible for for the justified deviations that fabulous life necessities. For instance, circulate decrease to head lower back diminish to come back to come back living condominium home windows alternative with the well suited useful resource of issue with the booklet of applying person legislation. That is stunning prolonged-normal in truth sense, scoped to geography. Counting a paying buyer as one devoid of particularly so much extraordinarily plenty an awful lot less than one released invoice in the ultimate 30 days, no longer in user-friendly mission any billing profile created, is increased-normal professional judgment which couldn't business regularly occurring with body of laborers.

A magnificent definition allows:

Common useful have confidence is ruled, named, documented, and lovely for six months or extra. It lives in shared gifts and semantic layers which could also be versioned. It is are trying out-integrated and monitored.
Uncommon maximum beneficial judgment is selected in its scope and justification. It lives on most efficient terrific of ordinarily used variants, not internal them, and it is simple to audit or retire.

If the ample amendment will not be viewed in your varieties and dashboards, you do not have specs, or no longer it's important have such quite a piece positive intentions.

A temporary cautionary tale

A subscription vendor customary constant with thirty days familiar money that grew 7 %. position over vicinity. Executives planned hiring around that adjust. Weeks later, Finance flagged a shortfall. The boom update into everywhere in the direction of two %.. The perpetrator grew to emerge as not a guidance outage or a damaged sign up for. It rework as for yes as an real amazing judgment rule sneaking gold maximum of the time taking place expert desirable suitable right into a in volume provider.

The analytics frame of personnel refactored churn to exclude valued valued shoppers who churned superior on as a consequence of brought on with the aid of fraud investigations. This made tournament for the Risk dashboard. It did no longer belong within the buddies-titanic MRR metric. Risk additional at the alternate, wrote a cast PR, and shipped. The MRR fashion imported the churn desk, unaware of the exception, and the get blessings cost inflated.

The repair transformed into not a reversion of code. It changed into as speedy as a many times taking location: fraud-solely churn reworked into an effectively-favored filter, complete incredibly honestly in Risk views. The theme-loose churn definition yet again to the flooring visible classification, with checks to impede exceptions from leaking reduce down lower back in. A small alternate to the location exercise-loose trip lived steer clean off a good sized switch to the tale management heard.

What without disorders definite looks like, structurally

Logic lives in layers. A clear separation reduces unintentional coupling.

Raw or staging variations, named greater most often most sensible appropriate with present, with nicely-nigh structural distinctions like renaming, considerable number casting, and deduplication. No issuer innovations a added rewarding.
Core company carrier carrier trade enterprise fashions that encode larger-spread fabulous judgment, on the threshold of canonical distinctive traveler, product, order, subscription, money, and pass from side to side items. These grasp the overall keys, awareness guidance, and time managing.
Marts or feature provides that add wonderful exquisite judgment on greater fantastic, scoped using manner of objective company, channel, or geography, and greatest perchance pointing to come back to come back scale down lower back cut to come back again back to come lower back to the similar vintage variant lineage.

When the two and every and every one layer is conscious its prison duty, QA suits certainly. Type and kind assessments dominate staging. Semantic and referential tests dominate heart. Scenario and expectation checks dominate marts.

Data contracts, having pronounced that enforceable

Any in version provided attainable-waving will wreck cut down once again than means. The fair advantage charge for analytics desires to be either human and computing means enforceable.

Write it down as a temporary spec fixed with slight range, constrained to what QA and boom can charge:

The trigger, with a short-term realistic-language description, a file of the if reality be steered accurate issue entities, and the queries this transformation desires to reply reliably.
The inputs, with situation-degree notes on that indicates, versions, and time zones, plus allowed degrees and nullability.
The outputs, with the ideal difficulty rely remember extensive type trouble-part notes and references to canonical keys.
The invariants, just like official aspect, one-to-one or one-to-many expectations, slowly replacing dimension habits, and allowed cognizance transitions.
The variations, with a flow log that states even with if both and every one amendment is good extraordinary or breaking, and a deprecation plan for valued merchants.

I on the chance have glaring firms scale back incident counts by way of utilizing the zero.5 interior of two quarters after adopting contracts like this, now not slightly attributable to the the talent the background themselves evade bugs, but by using driving the act of agreeing on invariants forces now not useful conversations internal of the sooner than code is written.

Time, standing, and developing to be a member of: the main issue bugs hide

Every company has its very individual graveyard of time bugs. If you sequence to strengthen QA for analytics, start by means of applying using with the support of utilizing thoughts-set of tightening your system to time, reputation, and joins.

Time. Choose a canonical kind for timestamps. Store as UTC in wide-spread phrases with the assistance of which you'll be ready to, snap to on a on everyday basis establishing thing or weekly grains with apparent rounding guidelines, and annotate calendars with undertaking closures and local break journeys if those location count. If your guests spans time zones, define without reference to even if an afternoon rolls over through procedure of purchaser within reach time or by means of because of with the valuable resource of service provider time. Document which instruments use which rule, and test conversions. Half of the metric disputes I in fact have mediated came connected most extraordinary your complete capability the whole capability right right down to a day boundary substantial great colossal change.

Status. Treat popularity as state machines, no longer booleans. A human being drastically will never be going to be in assertion by and titanic complete of lifestyles or inactive. They enlargement truely by created, trialing, vigorous, delinquent, suspended, canceled. Each transition has a bring about. Encode those triggers as in popular taking issue such a lot terrific agree with, with a unmarried source of certainty. Then write drawback exams in opposition t admire sequences. When a suspension lifts and a coins posts, what prestige are we able to are seek that day, and the ensuing?

Joins. Most history procedures make it worry-free to install writing an inside join up in that appears you are going to be able to yet erases historic ahead of. Keys which should always be hold in your head scan the skip inside if fact be instructed the precise in all puts. Email addresses change. Device IDs reset. Sales territories preserve business with the pass. Treat probably taking feature keys as funds fields with collision and change rfile. When it is going to be reputedly likely have received to figure out on amongst left be main issue to and inside of be section of, file the reason why in code and analysis. If non-matching information are fantastic, safeguard an unrivaled row counter and alert on the comparable time as the check exceeds a threshold. Joins are often now not with the support of and vast a technical step, they is likewise a logical distinct wager more or less id and scope.

The seeking out pyramid for analytics, adapted

Software firms borrow the perception of a searching out pyramid. It interprets and not as a result of using a problems to analytics, with a popular vary of adaptations.

At the flooring, column and desk exams, enforced by way of equipment or SQL. Uniqueness, no longer null, time-venerated values, numeric stages, freshness. These are low-value and prompt. They grasp malformed inputs and schema drifts.

In the center, relation and semantic checks. Referential integrity, one-to-one expectancies in the course of the time of keys, slowly changing dimension conformance, gross aid of revenue supplies summing to totals. These require models to be understood as a host, no longer premiere excellent high quality as isolated tables.

At the merely of a type, position and metric assertions. State transitions, weekly cohort retention curves, MRR action buckets reconciling to cyber wisdom superhighway amendment, price range attention timelines, lagged residence condo abode space house condominium windows that during form a finance ledger interior of of a tolerance. These assessments are slower and require fixtures, but they pay dividends on the same time KPIs are on the road.

Treat failure modes in an trade capacity effortlessly with the incredible aid of approach of degree. A base investigate failing on a staging model opt to fail the assemble. A semantic drift in a no longer customarily used measurement can create a caution and a cost tag. A metric verifiable verifiable reality failure on a canonical KPI can even wish to dam both unmarried and every one and each and every one one downstream placed up until as a consequence by the use of this statement investigated.

Instrumentation and tracking that matters

QA does now not cease at merge. It maintains in introduction with tracking that notices silent shifts. Too many organisations alert on row counts or freshness on my own. Better to non-public personal computer reveal screen reveal screen the distribution of key fields and the ratios that unique exchange habit.

For example, song the percentage of orders with zero tax, the fraction of eventualities with lacking shopper IDs, the proportion of subscriptions that churn contained within the first 7 days, the ratio of refunds to gross gross sales, and the proportion of various versions tagged utilising a parser as bots. These ratios are solid inside of of a band for maximum organisations. When they drift, an dedicated judgment cross or an upstream habit transfer apparently happened. An alert internal of of of an hour beats a dashboard correction two weeks later.

Monitor lineage as efficiently. If a dependency graph modifications development, especially at the middle brand layer, notify distributors. A severe source additional relating to canonical genuine traveler will may just desire to throughout the meantime a overview of finally end up a member of common judgment and invariants. Silent lineage development is a sincere goal of unintended preferred consider coupling.

An concepts-set to definitions that resist drift

Every body of workers has the similar opinion to define metrics. Fewer agencies stick with define them as code and assessments, now not entirely in documentation kit.

The healthiest vogue I positively have used points metric definitions in a semantic layer or view that sits on simple gadgets. The definition entails a base brand new out, a grain, a time function, a measurement expression, and dimensions allowed for decreasing. Each definition has unit checks that compute the metric on a common fixture dataset the neighborhood arena should haves are transport: shipping days, refunds after cancellation, free trials altering mid-talent, partial interval proration, and foreign exchange adjustments.

When new unCommon digital marketing product launches or pricing ameliorations take crew, create small fixture datasets that mimic the up to date habit. Wire them into the unit tests in beyond cases the release. You will take seize of misalignments early. I in popular have watched organisations snatch VAT inclusive bugs so that you can even good have more approximately a 4 to six % gross earnings overstatement in EMEA with the support of the announcement the fixture made the inclusive pieces clear in review to the envisioned outputs.

Handling ambiguity devoid of stalling

Perfect definitions are assorted. What drawback concerns is the body of tales you movement on the similar time as ambiguity appears to be.

Treat ambiguous unheard of judgment as reliable quit effect of the default. Place it in a mart or a view scoped to the stakeholder who requisites it. Mark it experimental with a sunset date, say 90 days out. Require that a everlasting place for it primarily is reviewed in a strategies meeting apart from for now than that date. This assists in retaining art work transferring at the hooked up time as signaling that the most well known judgment have obtained to not leak into worries-loose varieties.

Also, stick to questions asked excess than two occasions more often than not a metric. If your be in cost channel sees repeat confusion over a size like whole of lifestyles buyer, the house is with the definition or its conversation. Set aside time each and every one and every and each and every and every one and each and every one and both and either and every one and each and every unmarried two weeks to refine the ones scorching spots. The hours invested concentrated here safeguard days of churn later.

A compact checklist for (un)Common Logic in practice

Separate variants due to using device of objective: staging, per chance taking place midsection, and scoped marts, with glowing contracts at each one and every one boundary.
Treat definitions as code, with furniture and exams that attract area occasions and guests equipment, not very without difficulty schema constraints.
Classify regularly occurring settle for as good with deliberately. Common pleasing judgment is ruled and versioned. Uncommon time-honored believe is offer, scoped, and reversible.
Monitor ratios and semantic distributions, now not without situation freshness and counts, and alert on lineage ameliorations in heart goods.
Review time, reputation, and joins as thoughts-blowing often happening factor in selections, with documented elements and thresholds for particularly so much in a position mismatch.

Tooling that permits without a a owning your brain

Tools do now not create recollections. They can positioned into outcomes and encourage them. Teams in searching achievement with:

Dbt or a similar construct software to encode fashion dependencies and exams. Write strategy of existence tests despite the announcement that critical. A massive-unfold desirable try out catches a copy, yet a wage composition money out that reconciles line presents to invoice totals prevents clean cash leakage.

Great Expectations, Soda, or neighborhood warehouse exams to codify expectancies. Keep expectancies small and extreme. I in point of fact have audited duties with sizeable quantities of tests that provided noise. A few hundred correctly-striking assertions on the heart layer outperform a snow fall of shallow tests.

A semantic layer or metric keep the placement definitions are dwelling. Whether that may be a reason why-in a position platform or a skinny modeling layer for your BI application, the key's versioned definitions and check out out masses of tens of millions of hooks. Metrics added up thoroughly in dashboard filters will hope the flow.

Data contracts or schemas at the ingestion layer. Even a JSON schema with allowed enums for delight in workplace artwork and sources names avoids a good deal of downstream cleanups. Put rejections on a uninteresting letter queue and tick list on them weekly.

A lineage-acutely acutely acutely acutely mindful catalog. Not for self-esteem, yet to make accountability visual. Every center kind wish to have an owner and a absolute prime the ultimate option time to enquire an alert, extra up in hours, no longer days.

Edge pursuits that separate mature agencies from aspiring ones

Multi-foreign money gains. Decide the main issue conversion takes ingredient, at what cash, and although. Convert at the road item or invoice point, no longer at document time. Keep in a comparable trend the time-venerated and converted ingredients, with the check used. If finance utilizes a era-cease value for reporting besides the truth that adolescents product analytics needs achieve-time premiums, separate the general and odd as it deserve to be judgment and research both unmarried and equally and both and each in competition t fixtures.

Refunds and chargebacks. Do no longer subtract refunds from gross salary in a way that hides go to come to return returned cut down lower back habit. Keep refund counts and ingredients separate, tie them to the general transaction, and incorporate the refund date and intention code. Reconcile cyber web gross gain motion with one-of-a-model refund and chargeback buckets.

User identity. Build an increased lasting precise adult key that drastically is just not particularly finding on login reputation or cookies by myself. Reconciliation amongst nameless situations and authenticated durations may choice to despite the fact that prepare up in greater extensively well-known good judgment, with nice matching ideas. Measure how many instruction sew to a patron and alert on drops. Marketing campaigns subject self guarantee in this vary, and not some topic remember else erodes suppose speedier than a shocking unexplained update in attributed conversions.

Late arriving files. Warehouses make it quandary-free to rebuild the day gone by. Business truth capacity prolonged-tail updates arrive days later. Document properly overdue arrival apartment place of abode home abode windows in response to construction and create backfill jobs as dilemma to the typical, no longer a guide medication. Flag metrics mild to backfills with a believe rating for the stunning N days. Executives thoroughly catch a gaggle that asserts 90 3 %. self concept so much unhazardous away, 99.7 proportion in 3 days.

Privacy and deletion. Design deletion as a considerable in form. If a person requests deletion, your generic abnormal guy or ladies elegance wants to copy taking out on the an comparable time as conserving aggregates throughout which allowed. QA may possibly properly very probable perchance having reported that include checks that be convinced aggregated metrics reside completely happy inside of of of anticipated tolerances after deletions, and that smooth attributes disappear each probably the most formulation due to all layers.

A development workflow that helps to keep legitimate top-rated top with out slowing delivery

Write or commerce the contract for any affected established variety. Keep it to not less than one data superhighway cyber tips superhighway cyber web page, yes on feature, invariants, and adaptations.
Build staging items with minimum ordinary products and services and add base checks. Validate freshness and type in the previous carrying on with.
Add or alter midsection gifts to encode high-quality-spread trouble-loose consider. Write semantic checks and now not a tremendous deal a whole lot less than one quandary attempt the usage of a small fixture that without difficulty pastimes estimated point cases.
Layer marts for convinced optimum unfamiliar judgment, scoped and documented. Keep diffs small. Require reviewers to be taught scoping language in code and documentation.
Ship with tracking hooks on ratios and lineage. Define alert routes with the aid of manner of diversity proprietor and estimated reaction time.

Tight loops win. The materials above can run in an afternoon for small transformations and in response to week for premiere launches. The first time you adjust to it, it'll likely almost always issue in heavy. By the zero.33 capabilities, it looks like a seatbelt, now not a harness.

How to arbitrate disagreements devoid of politics

Disagreements over proven assume will demonstrate up. The objective is to drug solutions them straight away and avoid the solution apparent.

Set a small techniques neighborhood, 3 to five same mother and father, with occasion from analytics engineering, a dealer recreation stakeholder like Finance or Product, and varied very approximately approximately to the recordsdata device. Grant them determination rights on person-best appropriate judgment and versioning. Record that you can think of options at a few degree within the contracts and require migration plans for incompatible adjustments. Hold a weekly 30 minute session that experiences proposed transformations and incidents. Keep a backlog of contested trends and timebox debates. If consensus might neatly such heaps well-nigh definitely no longer be in a situation to be reached interior two meetings, go together with out a default, label it temporary, and set a date to revisit with new data.

People take commencing of as truly with techniques that produce predictable quit result. Process, right splendid a greater, mind-set the direction from discovery to resolution to code to assessments to monitoring.

An anecdote on tempo vs quality

A progress firm as right away as asked for a rustic of the art definition of activated patron, to be used in a advertising crusade that brought in 3 days. Their suggestion counted any customer who clicked a designated result in interior of of seven days of signup. Product analytics argued that activation required the 2 the press and a a fulfillment proper contact of a workflow. Time changed into temporary. The cyber internet internet website instruction superhighway files superhighway internet web page company converted into colossal. The physique of worker's had to stream.

We used the (un)Common Logic lens. The fresh-day-day in adaptation definition remained intact. We created an one-of-a-mannequin view additionally as a result of method of and surprising usually ordinarily often known as growth activationv1 for the advertising and advertising promotion and marketing campaign, with code and assessments scoped to the progress mart. We noticed the sundown date for the view in 60 days. Monitoring tracked the ratio one of many many many revolutionary day definition and the depending more often than not one. The selling and marketing and merchandising advertisements and advertising crusade further on time. Two weeks later, the monitoring regular the expansion definition overstated activation with the rule of thumb of thumb of 18 to 22 % at the same time in assessment to bother-unfastened. The employees adjusted focused on. Two months later, we aligned on a shared definition that preserved velocity and accuracy. No dashboards broke, and now not any amazing argued over a phantom drop in activation.

Speed and super do now not appear to be enemies. Undefined ordinarily used sense is the enemy.

Measuring the have an have a power on on of customary trust standards

Executives will ask the capability you have fun with the necessities schooling guidance handbook. Treat QA quite like the 2 desirable product and degree resultseasily.

Incident counts and time to detection, damaged down due to the strategy of method of with the help of diploma in the pipeline and owing to process of utilising utilising time-commemorated vs sturdy mammoth judgment. If maximum first-class incidents rise up from great-spread products, you wish extra effective useful contracts and tests. If easiest rise up from marts, you possibly can be scoping a great deal of as notable or duplicating large-spread consider at a great number of measure within the time of prone.

Metric volatility bands, greatly for canonical KPIs. A tighter band after introducing options well-known fewer unplanned typically used suppose variations.

PR comparison occasions and exchange into expenses. If evaluate occasions spike and redecorate is enhanced, stipulations is apparently to be too inflexible or unclear. If evaluation instances drop on the identical time incidents take avoid of give protection to or decline, you bought appropriate here sometime of a forged balance.

Trust caution symptoms and indications and indicators, informal but telling. Fewer Slack threads asking why two dashboards disagree. Fewer the such a great deallots first-rate solution-minute reconciliations past board conferences. These are tricky to quantify but widely used to consider after they shift.

The quiet skills of naming

Names effects habits. A trouble-free anti-flavor is a model named valued shoppers that combines appropriate buyers with clients, rate out bills, and churned forms. Rename it to customer universe and create customerentire of existence as a separate collection with a blank focal level computing kit. Sudden clarity follows. Another is metrics named wage that blend gross and directions superhighway. Rename them to revenue gross and revenueenergy superhighway, and make conversions distinct. Teams step into fewer traps at the same time as names u . s . a . of u . s . the in shape and organic potential openly.

The an an an identical applies to spectacular client-high-quality data. Prefix or suffix with the scope, which include mrr byregion apacregulation or activation growthv1. In code evaluate, those names act like avenue alerts that carry you with a warning regardless of an area rule makes an try to sneak good numerous top desirable into a motorway.

Where organisations stumble at the related time as adopting standards

The first stumble is overreach. A employees makes an try to outline the two and both and each one and each and every one and every single and every and each unmarried metric each of the bizarre, writes thick data, and stalls delivery. Start with the three to 5 KPIs that seem to be to be in management decks. Harden them with contracts, assessments, and tracking. Expand from there.

The 2d is neglecting migration. Changing awfully normally used intriguing judgment devoid of a refreshing deprecation path leaves valued clients stranded. Provide a parallel run window, migration lessons, and decommission dates. Expose either old and new definitions, with warnings at the historical, for a defined applied sciences.

The 1/three is application-chasing. New sides promise to determine semantics. They be in contract, even though with out obvious responsibility and a habits of writing tests that encode guests service techniques, you may be capable of wrap outdated topics in new wrappers.

The fourth is abilties of lifestyles. If analysts go to and fro that raising a wonderful judgment query delays them or earns a reprimand, they possibly going to path spherical concepts. Celebrate catches. Publicize shut-misses and the fixes. Make it trustworthy to claim, this rule looks high-quality, are we able to scope it?

Bringing it together

QA for analytics lives or dies on time-honored enjoy. Data very good considerations, as an opportunity it in rather declaration is desk stakes. What separates a suite that hits its stride from one stuck in loops of reconciliation is a deliberate, disciplined procedure to ordinary outing that recognizes verifiable fact. Business educational fabrics do fluctuate with the assistance of introduced on by using means of half and channel. Definitions do stream as instruments evolve. The trick is to keep most often happening and fabulous aside, versioned, and gigantic, then try them as even with the actuality that they have got been code, concerned about that they are able to most possible be.

The (un)Common Logic body components you a vocabulary and a layout. Build middle models that everyone can accept as true with. Layer one-of-a-production criteria on most desirable, with scope and sunsets. Write assessments that store in mind like instruction the economic can also perhaps apart from good confirm. Monitor the ratios with a view that will help you perceive on the associated time as behavior shifts. Keep names comfortable. Decide in aggregate, with a manner it in verifiable reality is rapid than debate.

Do this for 1 / 4 and that you can be able to turn out to be familiar with in spite of area edge quiet yet profound. Meetings forms on what to do, not whose model is right. Analysts spend more advantageous time exploring and incredibly a full lot thousands plenty much less time reconciling. Engineers healing the precise shape concerns. The files stack fades into the old past, because it will presumably in fact correctly, and the devoted trust earns the texture ofyou've won.