[av_section min_height='' min_height_px='500px' padding='default' shadow='no-shadow' bottom_border='no-border-styling' scroll_down='' id='' color='main_color' custom_bg='' src='' attach='scroll' position='top left' repeat='no-repeat' video='' video_ratio='16:9' video_mobile_disabled='' overlay_enable='' overlay_opacity='0.5' overlay_color='' overlay_pattern='' overlay_custom_pattern='']
[av_heading heading='Type I and Type II Error, Smoke Detector and the Boy Who Cried Wolf' tag='h3' style='blockquote modern-quote modern-centered' size='30' subheading_active='' subheading_size='15' padding='0' color='' custom_font=''][/av_heading]
[/av_section][av_section min_height='' min_height_px='500px' padding='default' shadow='no-shadow' bottom_border='no-border-styling' scroll_down='' id='' color='main_color' custom_bg='' src='' attach='scroll' position='top left' repeat='no-repeat' video='' video_ratio='16:9' video_mobile_disabled='' overlay_enable='' overlay_opacity='0.5' overlay_color='' overlay_pattern='' overlay_custom_pattern='']
[av_two_third first min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_textblock size='' font_color='' color='']
A/B testing is an essential component of large scale online services today. So essential, that every worth mentioning online business is already doing it. A/B testing is also used in email marketing by all major online retailers. Obama for America data science team received a lot of press coverage for leverage data science, especially A/B testing during the presidential campaign.
Here is an interesting article on this topic: http://kylerush.net/blog/optimization-at-the-obama-campaign-ab-testing/
If you have been involved in anything related A/B testing (online experimentation) on UI, relevance or email marketing, chances are that you have heard of Type I and Type II errors are. The usage of these terms is very common but a good understanding of these terms is not as common.
I have seen illustrations as simple as this. [Source: http://allpsych.com/]
[/av_textblock]
[/av_two_third][av_one_third min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_image src='http://datasciencedojo.com/wp-content/uploads/type1and2error.gif' attachment='28970' attachment_size='full' align='center' styling='' hover='' link='' target='' caption='' font_size='' appearance='' overlay_opacity='0.4' overlay_color='#000000' overlay_text_color='#ffffff' animation='no-animation'][/av_image]
[/av_one_third][av_one_third first min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_image src='http://datasciencedojo.com/wp-content/uploads/wpid-boy-31-300x183.jpeg' attachment='28971' attachment_size='full' align='center' styling='' hover='' link='' target='' caption='' font_size='' appearance='' overlay_opacity='0.4' overlay_color='#000000' overlay_text_color='#ffffff' animation='no-animation'][/av_image]
[/av_one_third][av_two_third min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_textblock size='' font_color='' color='']
I intend to share two great examples I recently read that will help you remember this very important concept in hypothesis testing.
TYPE I ERROR: An alarm without a fire. TYPE II ERROR: A fire without an alarm.
Every cook knows how to avoid Type I Error – just remove the batteries. Unfortunately, this increases the incidences of Type II error. :)
Reducing the chances of Type II error would mean making the alarm hypersensitive, which in turn would increase the chances of Type I error.
Another way to remember this is by recalling the story of the boy who cried wolf.
[/av_textblock]
[/av_two_third][av_one_full first min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_textblock size='' font_color='' color='']
Null Hypothesis: There is no wolf.
Alternate Hypothesis: There is a wolf.
Villagers believing the boy when there was no wolf (Rejecting null hypothesis incorrectly): Type I Error
Villagers not believing the boy when there actually was a wolf (Rejecting alternate hypothesis incorrectly): Type II Error
Tailpiece:
The purpose of the post is not to explain type I and type II errors. If this is the first time you are hearing about these terms, here is the wikipedia entry: Type I and Type II Error.
[/av_textblock]
[/av_one_full][/av_section][av_section min_height='' min_height_px='500px' padding='default' shadow='no-shadow' bottom_border='no-border-styling' scroll_down='' id='' color='main_color' custom_bg='' src='' attach='scroll' position='top left' repeat='no-repeat' video='' video_ratio='16:9' video_mobile_disabled='' overlay_enable='' overlay_opacity='0.5' overlay_color='' overlay_pattern='' overlay_custom_pattern='']
[av_one_full first min_height='' vertical_alignment='' space='' custom_margin='' margin='0px' padding='0px' border='' border_color='' radius='0px' background_color='' src='' background_position='top left' background_repeat='no-repeat' animation='']
[av_heading heading='Discussion' tag='h3' style='blockquote modern-quote modern-centered' size='' subheading_active='' subheading_size='15' padding='10' color='' custom_font=''][/av_heading]
[av_comments_list]
[/av_one_full]
[/av_section]