I have the below code
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options
opts = Options()
opts.set_headless()
assert opts.headless # Operating in headless mode
browser = Firefox(options=opts)
txt=browser.get('https://duckduckgo.com')
print(txt.text)
However, when i execute it, i get the error "AttributeError: 'NoneType' object has no attribute 'text'" I think I should get the text of the HTML.What am I doing wrong?
This error message...
AttributeError: 'NoneType' object has no attribute 'text'
...implies that the you have tried to extract the text attribute from a NoneType object.
get(url) method loads a web page in the current browser session and doesn't returns anything.
Though you have tried to assign the variable text with the return value of get() as in:
txt=browser.get('https://duckduckgo.com')
txt still remains 'NoneType'
A bit more information about your usecase would have helped us to construct an answer in a better way. However if your are trying to retrieve the HTML you can use the page_source property as follows:
Code Block:
from selenium.webdriver.firefox.options import Options
from selenium.webdriver import Firefox
opts = Options()
# opts.set_headless()
opts.headless = True
assert opts.headless # Operating in headless mode
browser = Firefox(options=opts)
browser.get("https://duckduckgo.com")
print(browser.page_source.encode("utf-8"))
Console Output:
b'<html class="js no-touch opacity csstransforms3d csstransitions svg cssfilters is-not-mobile-device full-urls has-search-focus" lang="en_US"><!--<![endif]--><head>\n\t<meta http-equiv="X-UA-Compatible" content="IE=Edge">\n<meta http-equiv="content-type" content="text/html; charset=UTF-8;charset=utf-8">\n<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=1">\n<meta name="HandheldFriendly" content="true">\n\n<link rel="canonical" href="https://duckduckgo.com/">\n\n<link rel="stylesheet" href="/s1771.css" type="text/css">\n\n<link rel="stylesheet" href="/o1771.css" type="text/css">\n\n\n\n<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" sizes="16x16 32x32">\n<link rel="apple-touch-icon" href="/assets/icons/meta/DDG-iOS-icon_60x60.png">\n<link rel="apple-touch-icon" sizes="76x76" href="/assets/icons/meta/DDG-iOS-icon_76x76.png">\n<link rel="apple-touch-icon" sizes="120x120" href="/assets/icons/meta/DDG-iOS-icon_120x120.png">\n<link rel="apple-touch-icon" sizes="152x152" href="/assets/icons/meta/DDG-iOS-icon_152x152.png">\n<link rel="image_src" href="/assets/icons/meta/DDG-icon_256x256.png">\n<link rel="manifest" href="/manifest.json">\n\n<meta name="twitter:card" content="summary">\n<meta name="twitter:site" value="@duckduckgo">\n\n<meta property="og:url" content="https://duckduckgo.com/">\n<meta property="og:site_name" content="DuckDuckGo">\n<meta property="og:image" content="https://duckduckgo.com/assets/logo_social-media.png">\n\n\n\t<title>DuckDuckGo \xe2\x80\x94 Privacy, simplified.</title>\n<meta property="og:title" content="DuckDuckGo \xe2\x80\x94 Privacy, simplified.">\n\n\n<meta property="og:description" content="The Internet privacy company that empowers you to seamlessly take control of your personal information online, without any tradeoffs.">\n<meta name="description" content="The Internet privacy company that empowers you to seamlessly take control of your personal information online, without any tradeoffs.">\n\n\n<link title="DuckDuckGo" type="application/opensearchdescription+xml" rel="search" href="https://duckduckgo.com/opensearch.xml?atb=v167-3__"></head>\n<body id="pg-index" class="page-index body--home body--onboarding">\n\t<script type="text/javascript">\nvar settings_js_version = "/s2472.js",\n locale = "en_US";\n</script>\n<script type="text/javascript" src="/lib/l113.js"></script>\n<script type="text/javascript" src="/locale/en_US/duckduckgo9.js"></script>\n<script type="text/javascript" src="/util/u344.js"></script>\n<script type="text/javascript" src="/d2613.js"></script>\n\n\n\n<script type="text/javascript">\n DDG.page = new DDG.Pages.Home();\n</script>\n\n\n\n\t<div class="site-wrapper site-wrapper--home js-site-wrapper">\n\t\n\t\t\n\t\t\t<div class="header-wrap--home js-header-wrap">\n\t<div class="header--aside js-header-aside is-hidden"><a class="header__button--menu js-side-menu-open" href="#">\xe2\x87\xb6</a><div class="header--aside__item showcase header__label"><span class="header__clickable js-hl-button" data-type="showcase"><span class="js-popout-trig" aria-haspopup="true" aria-label="Check out the list of things that we\'ve also made." role="button" aria-pressed="false"><span id="wedonttrack">Privacy, simplified.</span></span><span class="popout-trig js-popout"><span class="js-popout-link js-showcase-popout ddgsi ddgsi-down" aria-hidden="true" data-type="showcase"></span><div class="modal modal--popout modal--popout--bottom-left modal--popout--sm js-popout-main" data-type="showcase"><div class="modal__box"><div class="modal__body"><nav aria-labelledby="wedonttrack"><section class="showcase__dropdown-top"><ul aria-label="Here are some things that we made that you might like."> <li class="fix showcase__dropdown__list"><a href="/app" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="app"><div class="woman-icon"></div></a><a href="/app" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="app"><h1 class="showcase__heading">Get Our App & Extension</h1><p class="showcase__subheading">Protect your data on every device.</p></a></li><li class="fix showcase__dropdown__list"><a href="/newsletter" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="newsletter"><div class="newsletter-icon"></div></a><a href="/newsletter" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="newsletter"><h1 class="showcase__heading">Subscribe to our Privacy Crash Course</h1><p class="showcase__subheading">Learn about online privacy right in your inbox.</p></a></li><li class="fix showcase__dropdown__list"><a href="https://spreadprivacy.com/tag/device-privacy-tips/" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="blog"><div class="privacy-simplified-icon"></div></a><a href="https://spreadprivacy.com/tag/device-privacy-tips/" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="blog"><h1 class="showcase__heading">Protect Your Devices</h1><p class="showcase__subheading">Check out our privacy device guides.</p></a></li><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/spread" class="eighteen js-hl-item" aria-hidden="true" data-type="showcase" data-id="spread"><div class="spread-icon"></div></a><a href="https://duckduckgo.com/spread" class="text-left showcase__link eighty js-hl-item" data-type="showcase" data-id="spread"><h1 class="showcase__heading">Spread DuckDuckGo</h1><p class="showcase__subheading">Help your friends and family join the Duck Side!</p></a></li></ul></section><section class="showcase__dropdown-bottom"><ul class="text-left" aria-label="We\'ve got even more things for you."><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/donations" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="donations"><div class="donations-icon"></div></a><a href="https://duckduckgo.com/donations" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="donations">$1,300,000 in privacy donations!</a></li><li class="fix showcase__dropdown__list"><a href="https://duckduckgo.com/traffic" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="traffic"><div class="traffic-icon"></div></a><a href="https://duckduckgo.com/traffic" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="traffic">Over 29 Billion anonymous searches.</a></li><li class="fix showcase__dropdown__list"><a href="https://donttrack.us/" class="eighteen showcase__icon js-hl-item" aria-hidden="true" data-type="showcase" data-id="dnt"><div class="privacy-tips-icon"></div></a><a href="https://donttrack.us/" class="text-left showcase__link eighty showcase__text js-hl-item" data-type="showcase" data-id="dnt">Learn why reducing tracking is important.</a></li></ul></section></nav></div></div></div></span></span></div><div class="header--aside__item social"><a aria-label="Twitter" class="header--aside__item header--aside__twitter js-hl-twitter header__clickable header__link" href="https://twitter.com/duckduckgo"></a><span class="popout-trig js-popout header__clickable header__label js-hl-button" data-type="social"><span class="js-popout-link ddgsi ddgsi-down" aria-hidden="true" data-type="social"></span><div class="modal modal--popout modal--popout--bottom-left modal--popout--sm js-popout-main" data-type="social"><div class="modal__box"><div class="modal__body"><div class="social__link"><a href="https://twitter.com/duckduckgo" class="js-hl-item" aria-hidden="true" data-type="social" data-id="twitter"><img class="social__icon js-lazysvg" data-src="/assets/icons/header/twitter-color.svg"></a><a href="https://twitter.com/duckduckgo" class="js-hl-item social__link__text" data-type="social" data-id="twitter">Twitter</a></div><div class="social__link"><a href="https://reddit.com/r/duckduckgo" class="js-hl-item" aria-hidden="true" data-type="social" data-id="reddit"><img class="social__icon--light js-lazysvg" data-src="/assets/icons/header/reddit-color.svg"><img class="social__icon--dark js-lazysvg" data-src="/assets/icons/header/reddit-dark.svg"></a><a href="https://reddit.com/r/duckduckgo" class="js-hl-item social__link__text" data-type="social" data-id="reddit">Reddit</a></div><div class="social__link"><a href="https://spreadprivacy.com" class="js-hl-item" aria-hidden="true" data-type="social" data-id="blog"><img class="social__icon--light js-lazysvg" data-src="/assets/icons/header/globe-alt.svg"><img class="social__icon--dark js-lazysvg" data-src="/assets/icons/header/globe-dark.svg"></a><a href="https://spreadprivacy.com" class="js-hl-item social__link__text" data-type="social" data-id="blog">Blog</a></div></div></div></div></span></div></div>\n</div>\n\t\t\t<div id="" class="content-wrap--home">\n\t\t\t\t<div id="content_homepage" class="content--home">\n\t\t\t\t\t<div class="cw--c">\n\t\t\t\t\t\t\t\t<div class="logo-wrap--home">\n\t\t\t<a id="logo_homepage_link" class="logo_homepage" href="/about">\n\t\t\t\tAbout DuckDuckGo\n\t\t\t\t<span class="logo_homepage__tt">Learn More</span>\n\t\t\t</a>\n\t\t</div>\n\n\t\t\t\t\t\t<div class="search-wrap--home">\n\t\t\t\t\t\t\t\t\t<form id="search_form_homepage" class="search--home js-search-form search--adv search--focus" name="x" method="GET" action="/">\n\t\t\t<input id="search_form_input_homepage" class="js-search-input search__input--adv" type="text" autocomplete="off" name="q" tabindex="1" value="" autocapitalize="off" autocorrect="off">\n\t\t\t<input id="search_button_homepage" class="search__button js-search-button" type="submit" tabindex="2" value="S">\n\t\t\t<input id="search_form_input_clear" class="search__clear empty js-search-clear" type="button" tabindex="3" value="X">\n\t\t\t<div id="search_elements_hidden" class="search__hidden js-search-hidden"><input type="hidden" class="js-search-hidden-field" name="t" value="h_"></div>\n\t\t<div class="search__autocomplete" style="position: absolute; display: none;"><div class="acp-wrap js-acp-wrap"></div><div class="acp-footer is-hidden js-acp-footer"><span class="acp-footer__instructions">Shortcuts to other sites to search off DuckDuckGo</span><span class="acp-footer__link"><a class="no-visited js-acp-footer-link" href="/bang">Learn More</a></span></div></div></form>\n\n\t\t\t\t\t\t</div>\n\t\t\n\t\n\n\t\t\t\t\t\t<!-- en_US All Settings -->\n<noscript>\n <div class="tag-home">\n <div class="tag-home__wrapper">\n <div class="tag-home__item">\n The search engine that doesn\'t track you.\n <span class="hide--screen-xs"><a href="/about" class="tag-home__link">Learn More</a>.</span>\n </div>\n </div>\n </div>\n</noscript>\n<div class="tag-home tag-home--slide no-js__hide js-tag-home"><div class="tag-home__wrapper"><div class="tag-home__item">The search engine that doesn\'t track you.<span class="hide--screen-xs"> <a class="tag-home__link js-tag-item-link" href="/spread">Help Spread DuckDuckGo!</a></span></div></div></div>\n <div id="error_homepage"></div>\n\n\n\t\n\t\t\n\t\t\t\t\t</div> <!-- cw -->\n\t\t\t\t</div> <!-- content_homepage //-->\n\t\t\t</div> <!-- content_wrapper_homepage //-->\n\t\t\t<div id="footer_homepage" class="foot-home js-foot-home"></div>\n\n<script type="text/javascript">\n\t{function seterr(str) {\n\t\tvar error=document.getElementById(\'error_homepage\');\n\t\terror.innerHTML=str;\n\t\t$(error).css(\'display\',\'block\');\n\t}\n\tvar err=new RegExp(\'[\\?\\&]e=([^\\&]+)\');var errm=new Array();errm[\'2\']=\'no search\';errm[\'3\']=\'search too long\';errm[\'4\']=\'not UTF\\u002d8 encoding\';errm[\'6\']=\'too many search terms\';if (err.test(window.location.href)) seterr(\'Oops, \'+(errm[RegExp.$1]?errm[RegExp.$1]:\'there was an error.\')+\' Please try again\');};\n\t\n\tif (kurl) {\n\t document.getElementById("logo_homepage_link").href += (document.getElementById("logo_homepage_link").href.indexOf(\'?\')==-1 ? \'?t=i\' : \'\') + kurl;\n\t}\n</script>\n\n\t\t\n\t\n\t<div class="nav-menu--slideout"><ul class="nav-menu__list"><i class="nav-menu__close js-side-menu-close">X</i><ul class="nav-menu--theme"><li class="nav-menu__heading"><span>Settings</span></li><li class="nav-menu__item"><ul class="nav-menu__themes"><li class="nav-menu__theme js-side-menu-theme theme-is-selected"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#084999;"></span><span class="nav-menu__theme-color-bot" style="background-color:#888;"></span></span></li><li class="nav-menu__theme js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#333;"></span><span class="nav-menu__theme-color-bot" style="background-color:#888;"></span></span></li><li class="nav-menu__theme js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#3434d4;"></span><span class="nav-menu__theme-color-bot" style="background-color:#047d00;"></span></span></li><li class="nav-menu__theme js-side-menu-theme"><span class="nav-menu__theme-color"><span class="nav-menu__theme-color-top" style="background-color:#444;"></span><span class="nav-menu__theme-color-bot" style="background-color:#222;"></span></span></li></ul></li><li class="nav-menu__item clear"><a href="/settings#theme" data-settings="1">More Themes</a></li><li class="nav-menu__item"><a href="/settings" tabindex="-1" data-settings="1">Other Settings</a></li><li class="nav-menu__item"><a href="/bangs">!Bang Search Shortcuts</a></li></ul><ul><li class="nav-menu__heading"><span>Privacy Essentials</span></li><li class="nav-menu__item"><a href="https://duckduckgo.com/">Private Search</a></li><li class="nav-menu__item"><a href="/app">App and Extension</a></li></ul><ul><li class="nav-menu__heading"><span>Why Privacy</span></li><li class="nav-menu__item"><a href="https://spreadprivacy.com/">Privacy Blog</a></li><li class="nav-menu__item"><a href="https://duckduckgo.com/newsletter">Privacy Crash Course</a></li><li class="nav-menu__item"><a href="/spread">Help Spread Privacy</a></li></ul><ul><li class="nav-menu__heading"><span>Who We Are</span></li><li class="nav-menu__item"><a href="/about">About Us</a></li><li class="nav-menu__item"><a href="/privacy">Privacy Policy</a></li><li class="nav-menu__item"><a href="/hiring">Careers</a></li><li class="nav-menu__item"><a href="/press">Press Kit</a></li></ul><ul><li class="nav-menu__heading"><span>Keep in Touch</span></li><li class="nav-menu__item"><a href="https://twitter.com/duckduckgo">Twitter</a></li><li class="nav-menu__item"><a href="https://reddit.com/r/duckduckgo">Reddit</a></li><li class="nav-menu__item"><a href="https://help.duckduckgo.com/">Help</a></li></ul></ul></div><div class="badge-link badge-link--top badge-link--fadein ddg-extension-hide js-badge-link"><a class="badge-link__wrap js-badge-main-msg" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi"><div class="badge-link__thumb"><img class="badge-link__thumb__img" src="assets/onboarding/robot-icon-frameless.svg"></div><p class="badge-link__title js-badge-link__title">Your data shouldn\xe2\x80\x99t be for sale.<br>At DuckDuckGo, we agree.</p><ol class="badge-link__bullets"><li class="badge-link__bullet"><span class="badge-link__bullet-num">1</span>Block advertising trackers.</li><li class="badge-link__bullet"><span class="badge-link__bullet-num">2</span>Keep your search history private.</li><li class="badge-link__bullet"><span class="badge-link__bullet-num">3</span>Take control of your personal data.</li></ol><div class="badge-link__btn-group"><span class="badge-link__btn btn btn--primary js-badge-link-button">Add DuckDuckGo to Firefox</span></div><span class="ddgsi badge-link__close js-badge-link-dismiss">\xc3\x97</span></a><div class="badge-link__wrap badge-link__wrap--hidden js-badge-cookie-msg"><div class="badge-link__cookie-msg"><p>Clear your cookies often?</p><p>Try our homepage that never shows these messages:</p><p><b><a href="https://start.duckduckgo.com">start.duckduckgo.com</a></b></p></div><span class="ddgsi badge-link__close js-badge-link-close">\xc3\x97</span></div></div><div class="onboarding-ed js-onboarding-ed"> <a class="onboarding-ed__arrow-teaser js-onboarding-ed-teaser" style="display: inline;"><img class="onboarding-ed__arrow-teaser__alpinist" src="/assets/onboarding/bathroomguy/teaser-2@2x.png"><div class="onboarding-ed__arrow-teaser__arrow-wrapper"><img class="onboarding-ed__arrow-teaser__arrow" src="/assets/onboarding/arrow.svg"></div></a><a class="onboarding-ed__arrow js-onboarding-ed-arrow" style="display: none;"><img src="/assets/onboarding/arrow.svg"></a><div class="onboarding-ed__slide onboarding-ed__slide-1 js-onboarding-ed-slide js-onboarding-ed-slide-1"><div class="onboarding-ed__content"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t store your<br data-owner="balance-text">personal information. Ever.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Our privacy policy is simple: we don\xe2\x80\x99t collect<br data-owner="balance-text">or share any of your personal information.</p><div class="js-onboarding-ed-button-small-1"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-1"><img src="/assets/onboarding/bathroomguy/1-monster-v2--pre-animation.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div><div class="onboarding-ed__slide onboarding-ed__slide-2 js-onboarding-ed-slide js-onboarding-ed-slide-2"><div class="onboarding-ed__content onboarding-ed__content--more-shadow"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t follow you<br data-owner="balance-text">around with ads.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t store your search history. We therefore have nothing<br data-owner="balance-text">to sell to advertisers that track you across the Internet.</p><div class="js-onboarding-ed-button-small-2"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-2"><img src="/assets/onboarding/bathroomguy/2-ghost-v2.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div><div class="onboarding-ed__slide onboarding-ed__slide-3 js-onboarding-ed-slide js-onboarding-ed-slide-3"><div class="onboarding-ed__content"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">We don\xe2\x80\x99t track you in or out<br data-owner="balance-text">of private browsing mode.</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Other search engines track your searches even when you\xe2\x80\x99re<br data-owner="balance-text">in private browsing mode. We don\xe2\x80\x99t track you \xe2\x80\x94 period.</p><div class="js-onboarding-ed-button-small-3"><a class="btn onboarding-ed__button-small ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-3"><img src="/assets/onboarding/bathroomguy/3-bathtub-v2--pre-animation.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"> </div></div><div class="onboarding-ed__slide onboarding-ed__slide-4 js-onboarding-ed-slide js-onboarding-ed-slide-4"><div class="onboarding-ed__close"><span class="js-onboarding-ed-dismiss onboarding-ed__close-dismiss">Dismiss forever</span><span class="ddgsi onboarding-ed__close-divider">|</span> <span class="onboarding-ed__close-back js-onboarding-ed-back-to-search">Back to search</span> </div><div class="onboarding-ed__content js-onboarding-ed-install"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">Switch to DuckDuckGo and<br data-owner="balance-text">take back your privacy!</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">No tracking, no ad targeting, just searching.</p><div class="js-onboarding-ed-button"><a class="btn onboarding-ed__button ddg-extension-hide" href="https://addons.mozilla.org/firefox/downloads/latest/duckduckgo-for-firefox/addon-385621-latest.xpi">Add DuckDuckGo to Firefox</a></div></div><div class="onboarding-ed__content js-onboarding-ed-spread" style="display:none"><h1 class="onboarding-ed__title js-onboarding-ed-balance-text" style="">Already a fan?<br>Invite friends to the Duck Side!</h1><p class="onboarding-ed__subtitle js-onboarding-ed-balance-text" style="">Share DuckDuckGo and help friends take their privacy back!</p><div><a class="btn onboarding-ed__button onboarding-ed__tweet-button js-onboarding-ed-tweet-button" target="_blank" href="https://twitter.com/?status=DuckDuckGo%20is%20my%20search%20engine%20of%20choice.%20%23ComeToTheDuckSide%20-%20we%20have%20privacy.%20https%3A%2F%2Fduckduckgo.com%2F%231">Tweet #ComeToTheDuckSide</a></div></div><div class="onboarding-ed__image-container js-onboarding-ed-image-container-4"><img src="/assets/onboarding/bathroomguy/4-alpinist-v2.svg" class="onboarding-ed__image--placeholder" width="1200" height="460"></div></div></div></div> <!-- site-wrapper -->\n\n\n<iframe id="iframe_hidden" src="/post2.html"></iframe></body></html>'
Here you can find a detailed discussion on What is a None value?