Discover and read the best of Twitter Threads about #spark
Most recents (19)
Learning data science on your own is tough...
...(ahem, it took me 6 years)
So here's some help.
5 Free Books to Cut Your Time In HALF.
Let's go! 🧵
#datascience #rstats #R
...(ahem, it took me 6 years)
So here's some help.
5 Free Books to Cut Your Time In HALF.
Let's go! 🧵
#datascience #rstats #R
1. Mastering #Spark with #R
This book solves an important problem- what happens when your data gets too big?
For example, analyzing 100,000,000 time series.
You can do it in R with the tools covered in this book.
Website: therinspark.com
This book solves an important problem- what happens when your data gets too big?
For example, analyzing 100,000,000 time series.
You can do it in R with the tools covered in this book.
Website: therinspark.com
2. Geocomputation with #R
Interested in #Geospatial Analysis?
This book is my go-to resource for all things geospatial.
This book covers:
-Making Maps
-Working with Spatial Data
-Applications (Transportation, Geomarketing)
Website: r.geocompx.org
Interested in #Geospatial Analysis?
This book is my go-to resource for all things geospatial.
This book covers:
-Making Maps
-Working with Spatial Data
-Applications (Transportation, Geomarketing)
Website: r.geocompx.org
[1/🧵] Is this you? 🤨
• Holder of #XRP since December 2020
• Has never heard of $FLR before (Hello, #Spark?)
• Has no idea what the fuss is all about
• Slept under a rock for two years
Allow me to bring you up to speed w/ this crash course on @FlareNetworks and $FLR. 🧵👇
• Holder of #XRP since December 2020
• Has never heard of $FLR before (Hello, #Spark?)
• Has no idea what the fuss is all about
• Slept under a rock for two years
Allow me to bring you up to speed w/ this crash course on @FlareNetworks and $FLR. 🧵👇
[2/20] If you've been following this topic, you'll recall that there was once a #XRPL snapshot:
• 12th December 2020, 00:00 UTC
• #XRP Ledger index 60,155,580
. . .
• 12th December 2020, 00:00 UTC
• #XRP Ledger index 60,155,580
. . .
[3/20] . . .
You may recall that you had to claim your #Spark coins by providing your #EVM-based $FLR address in the message key field of your #XRPL account.
→ claim period expired on #XRPLedger index 64,204,315
→ Message could be changed until 11th June 2021, 23:59 UTC
. . .
You may recall that you had to claim your #Spark coins by providing your #EVM-based $FLR address in the message key field of your #XRPL account.
→ claim period expired on #XRPLedger index 64,204,315
→ Message could be changed until 11th June 2021, 23:59 UTC
. . .
Free SQL Resources for beginners!
Spend $0 and earn yourself some SQL knowledge with these sites.
Post Credit - Travis Tang Sir
🧵👇
Spend $0 and earn yourself some SQL knowledge with these sites.
Post Credit - Travis Tang Sir
🧵👇
1️⃣ 𝙆𝙝𝙖𝙣 𝘼𝙘𝙖𝙙𝙚𝙢𝙮's Intro to SQL: Querying and Managing Data comes with video lessons and interactive coding exercises.
(lnkd.in/g5y_Pfvf)
(lnkd.in/g5y_Pfvf)
2️⃣ 𝘾𝙤𝙙𝙚𝙖𝙘𝙖𝙙𝙚𝙢𝙮 provides a treasure trove of text tutorials for beginners and intermediates alike. (lnkd.in/gZ5_uYjC)
Today we are glad to announce these exciting features! 🎉
* Improved StatsForecast class
* Selection of best-performing models
* New documentation
* Plotting functionalities
Stay tuned for an exciting polemic comparison tomorrow!
#reinventforecasting
* Improved StatsForecast class
* Selection of best-performing models
* New documentation
* Plotting functionalities
Stay tuned for an exciting polemic comparison tomorrow!
#reinventforecasting
StatsForecast class:
With the new StatsForecast class, you can fit many models for many time series in less than 15 lines of code
You can include @dask_dev, @raydistributed, and #spark by changing just the backend parameter
With the new StatsForecast class, you can fit many models for many time series in less than 15 lines of code
You can include @dask_dev, @raydistributed, and #spark by changing just the backend parameter
Models: StatsForecast 27 models.
They can be included and imported in a few lines of code.
They can be included and imported in a few lines of code.
Today at @nixtlainc: fast and interpretable forecasting for multiple seasonalities. ⏲️🔎
This is exciting news for data scientists in the Python ecosystem. 🧵
#reinventforecasting
This is exciting news for data scientists in the Python ecosystem. 🧵
#reinventforecasting
Time series are the operational DNA of the world. 🧬
Most business data (sales, demand, electricity load, sensors) is stored in tabular format with time indexes.
Most forecasting models in Python are only suited for single seasonal data. ⚠️
#reinventforecasting
Most business data (sales, demand, electricity load, sensors) is stored in tabular format with time indexes.
Most forecasting models in Python are only suited for single seasonal data. ⚠️
#reinventforecasting
However, many time series have multiple seasonalities.
Multiple seasonality is traditionally present in data sampled at a low frequency. 💡
For example, hourly electricity data exhibits daily and weekly seasonality. 🎄🕎❄️
#reinventforecasting
Multiple seasonality is traditionally present in data sampled at a low frequency. 💡
For example, hourly electricity data exhibits daily and weekly seasonality. 🎄🕎❄️
#reinventforecasting
@nixtlainc started last year as a side project
Today we reached 1 million downloads🎉
Our goal is to shake the time series industry and make state-of-the-art algorithms available for everyone 🔥
This is how we got there
🧵1/11
#reinventforecasting
Today we reached 1 million downloads🎉
Our goal is to shake the time series industry and make state-of-the-art algorithms available for everyone 🔥
This is how we got there
🧵1/11
#reinventforecasting
@nixtlainc's ecosystem consists (now) of 5 #python libraries
Focus: speed, scalability, and accuracy 🚀
Some features:
* @scikit_learn syntax
* Native support for #spark, @raydistributed, and @dask_dev
* Models! Eg #arima, #LightGBM, #NeuralNetworks, #transformers 🤖
2/11
Focus: speed, scalability, and accuracy 🚀
Some features:
* @scikit_learn syntax
* Native support for #spark, @raydistributed, and @dask_dev
* Models! Eg #arima, #LightGBM, #NeuralNetworks, #transformers 🤖
2/11
We are really proud of the open-source adoption
Repos from Amazon, Mozilla, and DataBricks use us as dependencies 🏄♂️
We have contributions from people working at H20, Microsoft, Google, Facebook, SalesForce, Oracle, Shopify, AT&T, Blueyonder, Stanford, MIT and UCL 🙏
3/11
Repos from Amazon, Mozilla, and DataBricks use us as dependencies 🏄♂️
We have contributions from people working at H20, Microsoft, Google, Facebook, SalesForce, Oracle, Shopify, AT&T, Blueyonder, Stanford, MIT and UCL 🙏
3/11
This is thread # 3 in a series exploring using Spark in R with the sparklyr package. You can find the others here:
# 1:
# 2:
1/17
# 1:
# 2:
1/17
Here's what you'll learn reading this thread:
1. How a regular left join works.
2. The sort-merge-join algorithm (the default way to join big data with Spark and sparklyr).
3. Broadcast joins.
4. Salted joins.
All directly in R. Let's go.
2/17
1. How a regular left join works.
2. The sort-merge-join algorithm (the default way to join big data with Spark and sparklyr).
3. Broadcast joins.
4. Salted joins.
All directly in R. Let's go.
2/17
Ever wondered how to manipulate big data with R?
A thread on using Spark in R👇🧵
#rstats #spark #datascience
A thread on using Spark in R👇🧵
#rstats #spark #datascience
Big data is any data which is too large to be loaded into memory all in one go. Ever tried to read a large dataset into R and everything fell over or slowed waaaaaay down? This is where Spark comes in.
1/18
1/18
Spark is an open source tool for processing big data. It splits data into partitions for processing to overcome RAM limitations and writes to disk when needed. It is complicated but Spark handles most of the difficult parts. And you can work with Spark directly from R.
2/18
2/18
Can you imagine serverless Spark + BigQuery together? 🤯
Forget about managing clusters and tuning infrastructure if your job is to focus on create business value.
👇
🧵1/6
#googlecloud #bigquery #spark #dataengineering
Forget about managing clusters and tuning infrastructure if your job is to focus on create business value.
👇
🧵1/6
#googlecloud #bigquery #spark #dataengineering
Why Serverless Spark?
💡 Developers can focus on code and logic. They do not need to manage clusters or tune infrastructure. They submit #Spark jobs from their interface of choice, and processing is auto-scaled to match the needs of the job.
🧵2/6
#googlecloud #bigquery #gcp
💡 Developers can focus on code and logic. They do not need to manage clusters or tune infrastructure. They submit #Spark jobs from their interface of choice, and processing is auto-scaled to match the needs of the job.
🧵2/6
#googlecloud #bigquery #gcp
💡 Data engineering teams do not need to manage and monitor infrastructure for their end users. They are freed up to work on higher value #dataengineering functions.
💡 Pay only for the job duration, vs paying for infrastructure time.
🧵3/6
#googlecloud #bigquery #spark
💡 Pay only for the job duration, vs paying for infrastructure time.
🧵3/6
#googlecloud #bigquery #spark
🧵1. #IPERN officially launched on #StPatricksDay 2021 so it's our 1st birthday today!🎂🚑☘ It's been quite a year - we've hosted 10 events featuring 49 speakers (31% International 🌍 65% Female 🚺). Presented at 3 conferences, secured 3 grants and launched 5 SIG! #IrishEMS
2. How it all began… we announced ourselves to the Twitter world on Feb 1st '21. In Ireland this is traditionally the start of Spring (#Imbolc) so a time for new beginnings. It’s also St. Brigid‘s Day our matron saint who Is renowned as a healer #IrishEMS tinyurl.com/2p8w2zwt
3. Our official #IPERN launch on #StPatricksDay 2021 featured a fantastic array of Irish researchers presenting "elevator pitches" of their work and also included best wishes from international friends & colleagues! tinyurl.com/2p92at99 #IrishEMS
I want to start a new thread story. Not sure how long it will be and I may use different prompts: #vss365 #vssmagic #vssnature etc etc ... I love reading about legends and folk lore, and in my fantasy writing I often create my own myths. So this will be just a fun exercise...
Discordで教えてもらったんだけど、新しいMEW使ってる場合は24のシードフレーズでメタマスク上に復元できるので、インポートとかせずに直で復元すれば無問題のようです。
つまり、いずれBifrost wallet使いたくて、シードフレーズがあるMEWのETHアドレスで #FLR のオプトイン申請した人は、メタマスクのETHアドレスで申請し直さなくてもOK。 #XRP #FLR #Spark #FlareNetworks
Bifrost wallet は接続にシードフレーズが必要で、秘密鍵では接続できないから、シードフレーズがない旧版MEWのETHアドレスとは接続できないよ〜ということです。(多分)
Assuming you're not using aliases, a plain text search in @obsdmd for "[[page]]" within quotations pulls up the backlinks for that page. Searching for ".png" will pull up all of your pngs. There's a lot of flexibility that comes from searching strings when everything is in markup
Here I have an embedded query to find all my notes directly dealing with DSLs (domain-specific languages). I get this by searching for file names that have DSL in them. In Roam I might have had [[A [[DSL]] speeds up the author and gives them the ability to write from anywhere]]
Here I'm trying out using namespacing in a @dendronhq style. I have an embedded search looking for pages that have Clojure and function in the title. It's not officially supported in Obsidian, but with searches embedded on pages, I'm able to do it quite flexibly
How can I start a career in #risk management in the financial services industry?
@GARP_Risk @PRMIA @RIMSorg @actuarynews @SOActuaries @CQFInstitute @CFAinstitute
@GARP_Risk @PRMIA @RIMSorg @actuarynews @SOActuaries @CQFInstitute @CFAinstitute
I believe they are many degrees that can be a good value for money, provided you know where you would like to work in the long run.
In the developing world markets, employers still don't appreciate highly technical education in risk, actuarial sciences, financial engineering, quantitative economics, machine learning, etc.
For them, an #MBA is a be-all and an end-all troubleshooter.
For them, an #MBA is a be-all and an end-all troubleshooter.
1/365 - Starting this thread with one of the first Motorcycles to be built in India. The @royalenfield #Bullet350. A Motorcycle which was initially built for the Indian Army but with time found patronage amongst varying genre of people achieving a cult status over the decades.
2/365 - After selling imported Vespas for a while, Bajaj Auto started manufacturing the #Bajaj150 under license in 1959. This was one of the first Scooters in India and certainly changed the 2-Wheeler scene as it was initially available at a significant premium in the market.
3/365 Automobile Products of India (API) was the 1st Scooter maker in India and manufactured the #Lambretta Li 150 Series 2 under license from Innocenti at Bombay in the 1960's. This scooter was very popular and made appearances in a few famous Bollywood movies too. #BikerLife
2/ ☝️Este snapshot será tomado el 11 de diciembre de 2020 a las 18:00 hora de la Ciudad de México, 21:00 hora de Argentina, (00:00 UTC). ⏲️
3/ Es importante que sepas que un snapshot 📸 es cuando se registran los balances de todas las wallets en un momento específico del tiempo. En este caso, se refiere a una “fotografía” de los balances XRP en toda la red de XRP.
#100daysofCPD is where @UoB_HEFi will be beaming a spotlight onto #teaching and supporting #learning in #HigherEducation #CPD opportunities especially for #Beacon #HEAfellowship applicants & @uob_pgche participants during #COVID19 times & beyond
#100daysofCPD: #Day1 is the @UoB_HEFi #Learning and #Teaching Gateway. Full of self-paced online courses on a range of #technology and teaching topics. #BestPractice #CPD Go to birmingham.ac.uk/university/hef…
#100daysofCPD #Day2: @matt_tel_brum shares this Resource - How to Keep Uninvited Guests Out of Your Zoom Event containing all you need to know about Zoom’s #settings when using for #teaching, #meetings or #socializing during #COVID19 & beyond blog.zoom.us/wordpress/2020…
So, my thread on how to choose databases for your web apps starts in 20 minutes time. Hope y'all are ready ?However, this time i would love it to be an interactive session where we can learn from each other about database preferences for specific web apps and the use case
#thread
#thread
This is what we want to talk about today. Take a really good look at the image below. Study it well. Also notice the words at the 3 vertexes of the triangle. CONSISTENCY, AVAILABILITY & PARTITION-TOLERANCE (Big English - but fear not, we shall demystify all and it'll all be easy)
If we also look at the image in the previous tweet on this thread we can see some popular databases placed at certain sides of the triangle. It is important to note also the positioning of some of a particular point on each side. We shall discuss this in more detail soon. #thread
@AdobeSpark #Feedback. Pinch-in & Pinch-out are not very effective interaction to resize the text boxes precisely in Adobe Spark Android Mobile App. They are good for Zoom-in & Zoom-out from Maps because it don't require precision.
Sometimes when I try to left/right align two text boxes, suppose Title and Content. The auto snap helps me to properly align them but as I lift my finger, the text box slightly displace. That is irritating. I need a tool to fine-tune the movement. This interaction is
for Android Mobile App.
I am not sure that Adobe Spark primarily focusing on Print Media or Social Graphic. As I use Spark mostly for publishing Social Graphic thus I think "Popular" and "Stand out in Social" must be swapped in this page spark.adobe.com/sp/projects.
I am not sure that Adobe Spark primarily focusing on Print Media or Social Graphic. As I use Spark mostly for publishing Social Graphic thus I think "Popular" and "Stand out in Social" must be swapped in this page spark.adobe.com/sp/projects.
