Tag

Expedition

Parley SnotBot Alaska: the view from behind the camera

By | Alaska, Ocean Alliance News | No Comments

In a previous post, Ocean Alliance CEO Iain Kerr recounted the the nail-biting events leading up to National Geographic’s Earth Live segment during which SnotBot collected whale blow from a humpback in Alaska on live TV. Now Alex Tate, producer of the Alaska segment of National Geographic’s Earth Live, gives us a look at the

 

On 9th July 2017, the small island of Kake in South East Alaska hosted a team from National Geographic hoping to make television history. Two years in the making, Earth live was to be an unprecedented wildlife documentary with cameras across six continents – from lions in Africa to bull sharks in Fiji – all filming in a single live two-hour show. National Geographic had chosen Frederick Sound as a location to film humpback whales as part of this live spectacle, and they had clearly done their homework. It’s one of the best places in the world to see these majestic giants, especially during the months of July and August; still the odds of the team completing its mission were stacked against them.

That’s because they weren’t here simply to film humpback whales, they were following a scientific team from Massachusetts-based ocean Alliance, which was using a drone called SnotBot to collect whale exhale, or snot, when these giants come to the surface to breathe. The sample can offer amazing insights into the biology and physiology of the whales, from DNA to hormones, data that in the past scientists could only get invasively by getting close to a whale and taking a biopsy. And National Geographic wanted to broadcast this revolutionary scientific method on TV.

Attempting to film SnotBot capture a live snot sample was to be a real technological challenge, one which included boats drones, and an airplane. In total there were four cameras, the primary drone (SnotBot) taking the sample, a second drone filming SnotBot, a cameraman on the boat filming the SnotBot drone pilot, and a camera on an airplane with a gyro-stabilized gimble to keep the shots steady. The plane also was acting as a relay to get the live images from the boat back to a satellite on land which had been set up at Point Macartney. From there the images were to be beamed up into space and then on to the main studio in New York, where all the other live feeds from around the world were sent.

The whole team just before the Nat Geo shoot.

The whole team just before the Nat Geo shoot.

Because the feeds from the cameras on the boat (the Glacier Seal) had to get back to the satellite on point Macartney, it meant the range of the filming team was restricted to effectively line-of-sight, so they weren’t able to explore the full length of Frederick Sound on their hunt for whales. In any other year this might not have been a problem, as usually the area is jam-packed full of whales. However, this year the whales were scattered and isolated, with individuals using it as a highway but few staying around to feed. This made the Earth Live mission even harder.

If the lack of whales weren’t enough to worry about, the weather also caused concern. On the morning of the live event on the 9th, the clouds rolled in and the rain began to shower down. Not only was that bad news for the drones (they don’t take kindly to a drenching), but more crucially it meant the plane could not take to the sky. Without the plane, the live feeds from the boat could not be beamed back to the satellite on Point Macartney and then on to the studio in New York. At midday, the team had to do a live rehearsal with New York with no drones, no plane and, to add insult to injury, no whales. It did not look promising.

But they say fortune favors the brave, and at 4 pm (8 pm in New York, the time the live broadcast started), the clouds had cleared, the sun was out, the plane was up in the sky, and drones were on standby, ready to be called into action. At 4:30 pm, the drone was deployed, a whale had surfaced some 300 feet in front of the boat. Iain Kerr, CEO of Ocean Alliance and chief drone pilot, carefully maneuvered SnotBot into position above the whale’s blowhole, and on the third attempt he struck the bullseye. As the whale surfaced and took one last breath, its exhalation covered the petri dishes on SnotBot, complete with all the scientific information the team was hoping for. The timing couldn’t have been more perfect, as, with a flick of its fluke, the whale dove and vanished from sight. And for national Geographic, they achieved a world first — humpback whale snot captured on live TV!

This was my first visit to SE Alaska and Kake, and I have to say that the community support we received undoubtedly helped to make this expedition successful. People always greeted us warmly and were quick to offer help. I hope my work will bring me back to Kake again one day.

Alex Tate
Producer of Earth Live

From Patagonia to Alaska

By | Ocean Alliance News | No Comments

In 1987, I met Dr. Roger Payne on a windswept beach in Patagonia, Argentina. I was with John Atkinson and Jean Paul Gouin. It was a fateful meeting for John and me, our lives were forever changed. I have made my life’s work Ocean Alliance’s mission, and as well as being my closest friend, John has been the aerial photographer for our annual Southern Right Whale Program (with ICB) since 1990. Risking his life every year to hang out of a small airplane and photograph these animals for an aerial census.

 

Iain Kerr and John Atkinson

John worked with us on our most recent trip to Alaska, here is his blog:

Having just returned from the September 2017 Parley SnotBot expedition to Alaska, many people have asked me, what was it like up there in Alaska? Not a simple question because Alaska is a vast, empty and incredibly beautiful place where Mother Nature still lives mostly un-tethered by the hand of man.

Andy Rogan and John Graham, two of the usual SnotBot team members were not available for this expedition, so Iain asked me to help. My job was to manage the drone systems, remotes, batteries etc, launch and then catch the drone when returning from a flight, along with keeping all the gear clean, dry and organized.

After flying from Toronto to Edmonton and then on to Seattle on two separate airplanes, I boarded a third airplane that flew north alongside the rugged British Columbia coastline and on into southeast Alaska. Finally, after picking up a few passengers in the village of Ketchikan, in the midst of a torrential downpour we landed in Juneau.

Iain arrived the next morning with the Intel and the Parley for the Oceans team. After a get-acquainted dinner at a local restaurant, Iain and I were up early to catch the eight-thirty Alaska Seaplanes flight on a Cessna Caravan floatplane that flew us and our nine cases of equipment over the snow-capped mountains and down the foggy Chatham Strait. I figured the cell phone service in Alaska would be terrible but in the midst of that flight, much to my surprise, my cell phone rang. My brother was calling and I was able to share the spectacular scenery with him as we flew on down the windswept coast.

Forty minutes after departing Juneau, we landed on calm waters by the isolated village of Angoon, population 400 and dwindling, located on Admiralty Island, a ninety mile stretch of tall trees and rocky coastlines that the pilot told us contained the highest density of brown bears in the world. Much to our disappointment, during that whole week, we never saw one bear. Maybe just as well.

Waiting for us dockside was the Glacier Seal, a sixty foot custom built tour boat that would be our home for the next week. Onboard was Captain Marc, first mate Nate, and chef Collette who made sure we ate like kings and queens for the entire week. As we stepped on board, thermoses of steaming hot coffee, fresh baked bread, assorted fruits and cheeses awaited us. There was ample table space for our equipment and eating area, below deck were the sleeping quarters, and upstairs there was a huge open platform from which we would search for the whales, and fly the drones over them.

Two hours after Iain and I arrived, a second airplane landed with the Intel and Parley group. Last to arrive in his thirty foot research boat was Dr. Fred Sharpe, founder and lead scientist for the Alaska Whale Foundation. I first met Fred back in 1994 when we were up in Alaska making the Imax film Whales. Between then and now, I had only seen him once, for five minutes at a marine mammal conference. He is a really gentle soul with lots of great stories and it was really nice to reconnect with him.

After everyone’s gear was loaded onboard, we cast off all lines and for the next week we motored up and down vast fiords, all the while looking for whales, and when we found them, everyone cheered as Iain flew the drone over the whale and through the exhaling breath.

There is no better way to speak of the caliber of the people you’ll find in Alaska other then telling you this. On our last day, the majority of our team departed on the completely full morning flight back to Juneau. Because of incoming stormy weather, the captain of our charter boat was anxious to begin his twelve hour journey back home. This left Iain, Fred and me standing on the dock in the rain with all of our gear and four hours to wait for the next seaplane. We stored the gear in Fred’s boat and walked over to the fishing boat fueling office at the end of the dock. Inside was a grey-haired gentleman who greeted us kindly and offered us a cup of hot coffee.

We accepted the coffee and asked. “We have four hours to kill and thought we might take a tour around the village. Is there a taxi we can call?”

The man replied, “There are no taxis here in Angoon. But you can use my pickup truck.”

Our jaws dropped.

“You don’t even know us.”

The man smiled kindly and said, “The winters are long up here in north country. You learn real quick about who you can trust and who you can’t.”
We took it as a real compliment when he handed us the keys to his truck and said, “Welcome to Alaska boys.”

Thank you John, for your part in making this trip the success that it was – on that front I need to borrow a Tesla for a quick drive down to Florida – offers?

All the best.

Iain

Parley SnotBot Alaska – the science

By | Ocean Alliance News | No Comments

Dear Friends,

I love these expeditions, but I will admit that when I get back home I am exhausted. There is nothing more exciting that being in the field with a good hypothesis, good equipment and a great team. In the field, you just focus on the work that you are doing. Typically in the remote locations that we go to we have poor to no cell contact so we are cut off from the world around us, living in our work bubble (which is unusual in today’s ever-connected world). We work dawn to dusk, go through the data, back up hard drives at night, and then start it all again the next morning.

When we started the SnotBot program we were fairly confident that the tool would work at some basic level, but we did not realize the myriad experiments, applications and opportunities that these drones would bring to the marine science table. Nor did we realize how lucky we would be with the partners that would come onboard to help us with collection and preservation protocols, analyzing the samples and processing the data. Analysis team leaders to date are Dr. Scott Baker, Dr. Shannon Atkinson, and Dr. Fred Sharpe.

The population studies that Dr. Baker and his team are doing with the DNA are amazing. To learn more about Scott’s work I encourage you to visit his website. Please note that the paper “ ‘Eve’ and descendants shape global sperm whale population structure” came, in part, from data collected during Ocean Alliance’s Voyage of the Odyssey.

We are over the moon to be working with Shannon Atkinson’s Lab. As critical as hormones are to mammals, many people don’t understand how they work, so we are lucky that Kendall Mashburn from Shannon’s lab has written up a piece on hormones that she calls, “So Whales Have Hormones, Too?” In their most simple capacity, hormones are tiny molecules that govern nearly everything every animal does. From sleeping to eating to the transitional changes of puberty or pregnancy, these tiny molecules are potent activators and pretty much run the show in all animals’ bodies. The endocrine system is the group of organs responsible for the production of hormones.

Hormones come in two major varieties, the steroids and the proteins. Steroids are the major settings for the body’s operation, and the proteins fine tune those settings. Some familiar steroids are testosterone, progesterone, estrogen, and cortisol. Steroids are synthesized by the body using a cholesterol backbone in the same way as in almost all living organisms.  The term for this is evolutionary conservation. The beauty of evolutionary conservation is that steroid hormones have the same basic structures and functions in a fish as they do in a human. The structural similarities are a boon to people who study wildlife endocrinology, as it means that the tools used by your local hospital or lab to analyze human steroid hormones are not only commercially available, but generally compatible with steroids produced by any other living organism.

What we don’t know about whales, particularly the great whales, is astonishing. How do we know how best to protect them if we don’t know how they function and under what conditions they function best?  How can we tell when something is wrong? Since hormones in whales, as in other mammals, dictate a physical reaction, they are an excellent place to start to understand things like pregnancy and responses to stress, food shortages or other anthropogenic influences. Hormones are potent and evoke a distinct response. After their work is done, they are metabolized and excreted. In some cases, hormones are broken down by the body and the important bits are recycled. In other cases, they are expelled the way you would get rid of any other bodily waste product.  That’s right — urine, feces and breath (snot)! These forms of excrement can be some of the most precious biological samples available to wildlife scientists. SnotBot has the potential to swoop in and collect the respiratory blow from the animal. This gives the wildlife endocrinologist the ability to determine real-time concentrations of hormones of free-ranging animals.

It will take some time to define the baseline physiology of these incredible animals, but each snippet of information is one step closer to helping us better understand them.  More importantly, understanding the physiology allows us to communicate how the whale functions to people responsible for their conservation.  In turn, natural resource managers can make conservation decisions based on the actual health of the animals. And while it is true that being among whales is the experience of a lifetime, a wildlife endocrinologist gets positively hormonal the second they carry those precious samples into the lab!

Thank you, Kendall, Shannon, Scott, Fred and your staffs and interns for bringing such value to the work that we do.

Onwards Upwards.

Iain

SnotBot Indian Summer

By | Ocean Alliance News | No Comments

Dear Friends,

While I am not sure of the title of this email is politically correct, it is definitely good for us in Alaska. The day we arrived in Juneau, it was bucketing down rain and blowing 30 knots — this is fall in Alaska after all. The last two days (after foggy starts) have been blue skies and sunny, 5 to 10 knots of wind and calm seas.  We have also had good whale interactions, not the big numbers of animals we see in the summer, but that is OK. The Parley SnotBot gods seem to be with us.

It’s great to be working with Ted, Bryn, and Javier from INTEL Labs for a second  time, following our July expedition. This expedition (beyond collecting snot) we are working to further develop their photogrammetry and volumetrics programs as well as train them to fly and collect data over whales. I have often stated that I believe this work, these tools, are replicable, so we are trying to walk the talk; we have now collaborated with biologists in two other countries (Argentina and Mexico) and left them with drones and instruction / data sheets as part of this philosophy. And now INTEL.

Most people don’t realize that the expedition part of our work is just the beginning. In many ways the real work starts when we get home and start analyzing the data. But that can also be the most frustrating time; there is nothing more disappointing than getting home and finding out you did not collect some key data point, or that you saw some unique behavior / data but did not realize it at the time and as a consequence could not act on it. The best analogy I can think of is the difference between the old tech of using a film camera and the current tech of digital cameras. I remember coming home from our whale camp in Argentina with 20 rolls of film, not really knowing what I had captured but hoping for the best. With digital photos you can look at them real time and know if you got the shot. This is the capacity that the team (and the technology) at INTEL is developing with us, and it is a game changer — we are able to make better use of our time and resources, which is good for the whales, good for us, and even good for the funders that are supporting this work.

It is incredible how far we have come since Parley for the Oceans, our founding partner, introduced us to INTEL and brought us together at the United Nations on World Oceans Day 2017 on June 8. Not only have we developed new tech, but we have put that tech to use. I am excited to report that we have already identified whales that we saw on our first Parley SnotBot expedition this summer as a consequence of looking at our data in real time, in the field, and we’ve been able to estimate how the animals are doing healthwise. If you compare the attached photos taken at very different angles you can get an idea of how good this tech is that the INTEL team are developing. Circumstantial evidence suggests that this has not been a productive summer food wise for Alaska’s humpback whales. We saw far fewer whales in Frederick Sound this summer than we did in summer 2016, so this is a great time to be bringing this new technology to bear.

I am sending this email from a sat phone, so apologies for fewer photos and a shorter blog.

Another first for the Parley SnotBot team and another great collaborative effort.

Onwards Upwards.

Iain

Parley SnotBot Alaska expedition: A team effort

By | Ocean Alliance News | No Comments

Dear Friends,

With all the different players involved in this Parley SnotBot expedition, it was truly a team effort that made it the success that it was.

Here are a couple of short blogs from two of the Parley SnotBot Alaska team:

John Graham has been on every SnotBot expedition to date; he is our MacGyver, keeping the machines running, charging batteries and cleaning the drone’s after every flight (DNA contamination). He also catches and launches the drones. Catching a drone does come with some risk, particularly when a machine does not want to respond to the controls. After over 500 drone launch and recoveries, I thought that we had our safety protocols pretty well established. John caught a drone that was not responding to the radio controller; when he did, the situation fell apart as the drone throttled up on its own and became a risk to the whole crew. John threw it to the ground, but not before some serious cuts on his hands. John was wearing a helmet, safety glasses and gloves, but these new drones are powerful machines. Quick thinking on John’s part definitely prevented a more serious accident, and we now have protocols in place in case this type of situation comes up again.


Second opinions count.
 My first instinct [after the drone mishap] was to stay back on shore while the crew headed out for our daily research collection. I had arrived at this decision by taking into account a couple of factors. First, there is the fact that my altered physical abilities, due to an incident earlier in the expedition that resulted in 17 sutures, had changed my role on the team, preventing me from being a 100% contributor. Secondly, the boat we are using is jam-packed with nine crew members, drones, tech gear, batteries, laptops, camera gear, and a makeshift research lab station. All this makes getting around a real challenge.
 
The subtraction of one warm body, mine in this case, would give a little needed extra room for others to move about freely.

My second opinion on this matter was given by Iain, our team leader, who highly encouraged me to stay on board, contribute in whatever capacity I could, and we would all just make the best of the limited space we have available.

Well, I went with the second opinion, and boy, was that the right move.


 
In previous days, the whale population in our research area was a bit on the sparse side. Not so on this day! Large pods/ groups of humpbacks were bubble net feeding. There was breaching, pec slapping, lob tailing, and of course, lots of exhaled breath condensate! Oh, did I mention orca, too. But not just orca, breaching orca!! What an amazing sight.

Of course, all the activity did make it a challenge to collect samples, but Iain was up to the task, running his countless flight time experience through its paces.

All in all, a very good day. Sometimes a second opinion gets you to listen to your gut and not the logical part of your brain. I am very thankful that I did not let this amazing life experience pass me by.

Many thanks to Iain, my gut, and the incredible marine life that occupy this world with us, for giving me a story worth blogging about.
 
Thank you, John.

 

Kelly Cates is new to our team; she is a PhD student in Dr. Atkinson’s lab who will be involved in the data analysis. She is also keen to adopt SnotBot for her own research interests. She has collected snot from blue whales in the past using a long pole reaching out from a small boat:


Southeast Alaska is arguably one of the most beautiful places in the world, yet her secrets are often cloaked in fog, low laying clouds and cloying rain. This is the day we started out with, misting rain that hugged your every curve, a sunless multi-gray hued sky and not a critter on the horizon. This is not the day we ended up with. Between a pod of orcas, voracious bubble net feeders and a sunset that rendered words useless, we had a pretty alright day. That is, we were full on until we had to head home for fear of running out of fuel. As a newcomer to the Ocean Alliance team, I was quietly impressed with the speed and ease with which samples were collected. Vocally quiet. Inside my head I was berating myself for ever having tried to use a 21-foot pole to collect blow spray — how barbaric.

 

Drones provide access to animals previously only dreamed of. The speed with which a drone can approach animals, collect a sample, return to boat and then be out sampling again was mind boggling. We collected five samples in the time it would have taken a boat carefully maneuvering to a whale to collect one sample. The trouble with boat approaches is that the whale absolutely knows you are there and are notorious for Houdinii-ng their way out of sampling range. Research with free ranging cetaceans is a lot like Southeast Alaska. You spend a lot of time looking at nothing, but when the sun comes out there is no place else you would want to be. Cetacean research conducted with drones is going to lead to a lot more sunnier days. 
 
Thank you, Kelly.

No blog would be complete without Christian Miller’s photographs, so I have added a few that I hope are appropriate to the above stories.

Onwards. Upwards.

Iain

Parley SnotBot Alaska expedition: Hard work, but we learned a lot!

By | Ocean Alliance News | No Comments

Dear Friends,

It’s not the best of times for science funding. So, there is enormous pressure when you are planning an expedition to set project goals too high and possibly try to collect too much data.

Would we do that?  Of course.  The goals of this expedition (in no order of priority) were to:

  •  Collect snot from a whale with our Parley SnotBot on live TV.
  •  Work with INTEL to develop an “in the wild” animal identification system and a volumetric’s system that can help us determine the health of a whale in real time.
  •  Further refine the protocols and tools for collecting snot with a drone.
  •  Test two new drones for collecting snot, the DJI Inspire 2 and the DJI Mavic Pro.
  •  Test a new snot collection system we call the Kendall/Graham Funnelator.
  •  Test EarBot with a new acoustic transmission system and amplifier.
  •  Test a Zenmuse FLIR camera over whales and try to record blow/body temperature.
  •  Collect at least 30 robust snot samples to be shared with our collaborators.
  •  Work with Dr. Baker and Dr. Atkinson’s labs to develop the appropriate collation and preservation protocols so that a variety of analysis can be conducted on the snot.
  •  Expand the number of species that we have collected snot from to further validate this technique.
  • In our case, we had nine and a half days to do this, three of which were with National Geographic, leaving us with six and a half days unencumbered on the water. Of course, we are in Alaska, so you have to count on at least two bad weather days. No pressure! (The next grant I write I am going include a budget for post expedition psychiatric counseling.) Looking back, we probably set the bar too high on this one, but that is another lesson learned.

Considering the above, how did we do? In the best of British understatements, I’d say, “Not bad at all.”

·         We collected snot from a humpback whale with the Parley SnotBot on a Live TV show broadcast nationally and internationally – A FIRST.

·         Thanks to our collaboration with INTEL, we identified an individual whale from a drone before the drone even made it back to the boat. We also set the stage for real time photogrammetry and volumetrics – A FIRST.

·         We collected snot from an orca; we had thought that orca blows would be too small and the drones too big, but we did it (with a small drone) – A FIRST.

·         We flew the DJI Zenmuse FLIR camera and attempted (we have to review this data) to record the blow and consequent body temperature of a whale – A FIRST.

·         We worked with four different organizations including marine mammal, oceanographic and technology institutions.

·         We successfully flew and collected snot using two new (for us) drones the DJI Inspire 2 and a Mavic Pro over whales – A FIRST.

·         We flew the EarBot (a drone that lands in the water near the whales and records their vocalizations) for Earth Live but they did not use the segment.

·         We tested / flew a new Snot Collection system – we call the Kendall/Graham funnelator

·         To date we have used plastic petri dishes to collect Snot. Dr. Atkinson’s lab suggested that there might be an issue with hormones sticking to the plastic, so we flew with glass petri dishes as well as plastic – A FIRST.

·         We trained staff and collaborators in over water and over whales, flight launch recovery and operations.

The weather seemed to be fighting us more on this trip than any we have done so far, but we realized that this was actually a benefit to the program, giving us a broader operations perspective. For example; what are the maximum wind conditions to collect snot in, and what is the best way to collect snot in windy conditions? Can we collect snot in the rain (we think so but it was hard to tell because the dishes were always wet). We will have to wait for lab analysis to answer these questions?

Last but not least, Kendall Mashburn from the University of Alaska, Fairbanks informed us that she positively identified hormones in the blue whale blows from our last expedition. She validated progestogen and cortisol and now she is looking to validate the existence of testosterone and aldosterone.  These hormones are primary reproductive and stress hormones, so this is great news.

At the end of the day it’s been a really hard-fought data expedition but it has also been one of the most productive, because of this we have learned a lot about our limitations and have realized how much more we still have to understand about this remarkable game-changing technology for whale research.

We are leaving with a long list of upgrades and problems to solve that will make this technology more effective and easier to use.  So much credit goes to the incredible Parley SnotBot Alaska team — Andy Rogan, John Graham and  Christian Miller; Fred Sharpe and Andy Szabo from the Alaska Whale Foundation; Ted Willke, Bryn Keller and Javier Turek from INTEL; Scott Baker, Shannon Atkinson, Kendall Mashburn and Kelly Cates form the University of Alaska, Fairbanks; and all of the Ocean Alliance home team. Thanks are also due to Alex Tate and all of the Plimsoll Nat Geo production team and the crew of the Glacial Seal. I am very grateful to Amy and Dylan for putting up with a mad man for at least the last two months (maybe longer). Last but not least I want to thank our amazing hosts, Tinker and Gary at the Keex Kwaan Lodge – You guys are the BEST.  Thanks also to Patti for the great food. More blogs and incredible Christina Miller photographs to come!

Best Fishes from foggy Alaska.

Iain

 

Parley SnotBot Alaska expedition: Parley x INTEL

By | Ocean Alliance News | No Comments

Dear Friends,

As you are all aware, these trips by the nature of what we hope to achieve are always an adventure. This trip has been about the most challenging that I have ever done, a live TV show, partnering with INTEL, and testing four new drone setups and two new drones in Alaska!?!

The whale gods have been on our side, and we had another first yesterday: we collected Snot from orca whales using a DJI Mavic Pro – just amazing.  Even I had doubts that we could do this, but there was Snot in the dish so another first for the Parley SnotBot.

Our little boat was certainly full with the Parley SnotBot team, Alaska Whale Foundation, and the INTEL team on board, but what a great group of people.

Following is a blog from our newest friend Ted Willke, Senior Principal Engineer and Director of the Mind’s Eye Lab INTEL.  Our thanks go out to Bryn Keller and Javier Turek.

Ted Willke, center, and Fred Sharpe, right, photobombed by Iain Kerr

Ted Willke, center, and Fred Sharpe, right, photobombed by Iain Kerr

 

Coming into a new collaboration is never easy, especially when it involves subjects as diverse as marine biology, drones, and artificial intelligence.  But it’s a necessary dance if you want the kind of innovation we’re after — the kind that happens when diverse scientific fields collide.
 
This is the dance our Intel Labs team found itself in with Parley and Ocean Alliance this past month.  Javier Turek, Bryn Keller, and I were introduced to Iain Kerr and his team by Parley for the Oceans at the World Oceans Day conference at the UN.  The question posed to us was: How can artificial intelligence (AI) advance Ocean Alliance’s mission and whale biology … in the next 30 days??!!  We soon realized that we had a lot to learn about whales and Parley SnotBots, and fast!  But it was also immediately clear that our research team had a lot to offer.
 
Simply put, AI technology aspires to imbue machines with cognitive skills, like visual recognition.  AI equipped Parley SnotBots would clearly be a game changer for OA, even given OA’s existing game-changing techniques.  Today’s Parley SnotBots have cameras that are used for piloting and video capture.  But there’s a strong desire to do more with this data.  As Fred Sharpe put it, “In the age of modern sensors, we’re in a data maelstrom.   The real action is in the downstream processing.” And we knew that the right AI could take it on.
 
The stage was set, and we had two seriously ambitious goals: 1) finding a way to identify whales using images transmitted by Parley SnotBots, and 2) calculating a whale’s relative body composite index, a measure of its energy reserves and condition, from streaming video.  Our team, with its background in computing and machine learning techniques, felt up to the task.

Ted Willke preps drones for the National Geographic Earth Live shoot.

Ted Willke preps drones for the National Geographic Earth Live shoot.

 

But you’ve got to understand: these are tough tasks for AI even with ideal data and carefully groomed algorithms running on machines back in our lab.  So trying to solve such problems in a completely uncontrolled environment (weather, water, whales, drones) on a small ship out on the rough sea is nuts. Trying to get something together in four weeks for the National Geographic Earth Live broadcast — TOTALLY BANANAS!
 
There were other difficulties that we didn’t completely understand until we got to Kake, Alaska.  Ships like the Alaska Whale Foundation’s Paula T are like drone aircraft carriers.  The pace is fast and the space is cramped.  Any new technology has to be unintrusive, field-friendly and fuss-free.  Otherwise, it’s going overboard!
 
To complicate things further, we were still hacking code as we arrived in Kake.  We had never run the whale ID algorithm on images taken by a drone or fully validated the volumetrics analysis.  We really didn’t know if this stuff worked.  And we had never integrated our systems with Iain’s.

Ted Willke and Javier Turek crunching code

Ted Willke and Javier Turek crunching code

 

We figured out how to set up what amounted to a small computer lab on a ship we’d never seen.  The Earth Live dress rehearsal on July 8 came and went.  We continued to sweat it out.  With the Nat Geo team breathing down our necks, we hacked and hacked.
 
Then it was show time.  I won’t recap the gut-wrenching Earth Live affair since Iain described it in his recent post.  But I will say that it was one of the most harrowing adventures I’ve ever experienced.  We wrapped up our development as the show began.  By the end, Iain’s team had pulled off a Parley SnotBot collection miracle and our algorithms had made a positive ID on the same whale before the drone landed — a scientific first! (See photo at beginning of post)

Fred Sharpe, Iain Kerr, Javier Turek, Ted Willke.

Bryn Keller, Iain Kerr, Javier Turek, and Ted Willke

 

You’d think things would’ve let up after the Nat Geo team said goodbye, but they haven’t.  With the show out of the way, we immediately returned to the primary research mission and our associated computer science research.  With just a few days to collect the data needed to power the research for the next few months (and before the next expedition!), it has not been easy.  As Bryn put it, “Writing code while tracking whales around at 35 knots (on occasion) over rough waters (a lot) is really interesting.”
 
Even though it’s been a grind and major adrenaline dump, our enthusiasm remains unhampered.  How could it be otherwise, with whales breaching, lunge feeding, and checking out our boat?  Quoting Javier as he watched a humpback lunge feed in Keku Strait just a hundred feet from our boat on the morning of our last day, “This is FREAKING AWESOME!!!”
 
We’ll be back and we’ll be packing more AI when we do.

The whole team just before the Nat Geo shoot.

The whole team just before the Nat Geo shoot.

 

Best Fishes from Alaska.

Iain

Parley SnotBot, Alaska expedition powered by Intel: Stunned in SE Alaska

By | Ocean Alliance News | No Comments

Dear Friends,

I hope that you got a chance to see us on Nat Geo’s Earth Live; for us it was a crazy emotional rollercoaster but not an experience we would have wanted to miss. Here is a YouTube link to a segment: We were featured in four segments during the show, but to us at least this was the most exciting.

It’s now about 30 hours later, we had a regular SnotBot data collection day today, but I have to say that I am still ABSOLUTELY STUNNED by all that happened yesterday. I still cannot process how it all came together so successfully! So here is the back story.

Per Sleepless in SE Alaska, when Alex Tate, the director, and I said “Let’s do this,” it was with an abundance of whales in mind and on my part a total ignorance as to what a live show entailed.

Just the amount of people and equipment involved was massive (see photo below of some of equipment cases, not personal bags, on the docks).

The Plimsoll Nat Geo team consisted of:

An airplane with a Cineflex camera.
A satellite truck at the end of a dirt road alongside a fully extended crane with an antenna on top.
A full satellite transmission team & equipment on the boat.
A director, cameraman, rigger, and assistant on the boat.
And of course Fred Sharpe and Andy Szabo, from the Alaska Whale Foundation.

The idea was that there would be four cameras running consecutively during our live portion of the show:
1. Live feed from SnotBot
2. Live feed of SnotBot. Christians drone (filming Iain’s drone) and what a bloody amazing job he did!  I just took off after the whale with no consideration of how hard it might be to follow and film me by drone, and he kept me in frame!
3. A cameraman filming us
4. Aerial shot of boat and drones from the Cineflex camera.

The process was also pretty complicated. We were plugged into live feed cables and microphones, the live feeds were then transmitted from the boat up to the airplane and then down to the satellite truck and then back up via satellite to NYC. And it all had to work seamlessly or else we would not be featured in the show (think lots of electrical connectors in the rain).

While I was blown away by the professionalism of the whole team, we had some issues :-}. Basically, the area in which we could look for whales was limited by the audio and satellite transmission capacity, the height of the ceiling of the plane, etc.  My estimate is that we were given a 5-square-mile area (or less) in which to find a whale.  When I got this information I really, REALLY began to panic; up until then we could not find a whale in 20 square miles let alone 5 miles, and by the way we had to do it at 4:30 pm!! INSANE. We did have one whale that we followed for over an hour, and each time it came up it only did one blow. Basically impossible to collect from.

Of course the weather was miserable, and it was predicted to get worse for the afternoon’s live event. We even discussed not doing a dress rehearsal earlier in the day in case the drones got waterlogged and crashed – resulting again in not being featured in the show (we did have one spare).

By 1 pm I was in full-on desperation mode, we had seen and approached a couple of whales, but they swam out of the coverage zone. The whales were just not there. So, the director said to me, “We will go live to you at 4:30 and you can launch SnotBot and then we will go back to NYC and then you have an hour and a half to collect Snot.  Maybe we will cut back to you live as you do this collection or maybe they will say ‘This just happened.’”

So we are all prepped on the top deck, I had a microphone on me (I was reminded not to swear) and they said “3,2,1 we are live……”
I said “There’s a whale. Lets launch the drone” (no whale…sorry). As I hope you saw, I gave the drone to Andy and he prepped it. I flew the drone out of Andy’s hands and I will remember this moment for the rest of my life………A BLOOMING WHALE SURFACED 500 FEET AWAY RIGHT IN FRONT OF ME – WHHHAAAAATTTTTTTT!  IMPOSSIBLE!!!!!

As they say, the rest is history, on the third try (the last blow) I got the sample. But how?

What the heck just happened?  A whale surfaced at the right time (almost to the second) at the right place (so I could see it while flying) and the crew in NYC who were going to cut away stayed with the shot to the bitter end through three blows. Millions of people were snotted!!!!

Again, HOW IS THIS POSSIBLE??? If there was a lottery in Kake, I’d be out buying tickets right now. This has to have been a million to one shot.

We have a remarkable SnotBot field team (in the photo above, the SnotBot field team from left: Andy Rogan, John Graham, Iain Kerr, and Christian Miller, with cameraman Scott Tibbles and the director of our segment, Alex Tate.). The staff at our headquarters and our Founder/President Roger Payne all worked hard to make this happen – so yes we worked hard and were well prepared, but I still don’t understand – HOW DID THIS HAPPEN?  And who/where is that whale, so can we thank him or her????

As my wife, Amy, watched the snot collection she thought that it was a prerecorded segment. How could you go directly from a commercial break to collecting snot from a whale LIVE???? I truly don’t know, but we did it!! (Do NOT ask me to do it again live).

And I forgot to mention, right before the shoot the rain stopped and the sun came out; 30 minutes after the shoot we were trying to film a segment for the edited evergreen copy of the show that will come out at a later date and the rain came down again in buckets and we had to scramble to get all of the equipment under cover – look for that in the evergreen copy.

We will be back out on the water the rest of this week, further developing the SnotBot protocols and collecting samples for our partners at Scott Baker’s lab at Oregon State and Shannon Atkinson’s lab at the University of Alaska. We are grateful beyond measure for this opportunity and for our collaboration with Parley, but I guess the reason we are all here is that we are grateful beyond measure for the whales.

The team may not take out the boat tomorrow to collect snot – we are going to experiment with walking on water!!

Next blog will be talking more about the amazing work that our INTEL team are doing.

STUNNED in SE Alaska.

Iain

Parley SnotBot x Intel Alaska expedition: Sleepless in SE Alaska

By | Ocean Alliance News | No Comments

Dear Friends,

While we always metaphorically expect the unexpected on an expedition like this, you really hope that it won’t happen to you.  Especially when you are working with a live television production.

The last few days have been incredibly tough, in 2 and a half days on the water we have probably seen less than twenty whales and a lot of those we only saw their far away blows.  Last year we had literally hundreds of whales around us, we even joked that we could have done the SnotBot work from the shore because there we so many whales. Of course, you expect to have good and bad years with more or less whales but hundreds one year and less than 30 the next? The whales are in Alaska of course……just not here.

Luckily we have a remarkable team, or else the situation could be even worse.
14 people from the Nat Geo / Plimsoll production team.
4 core SnotBot team.
2 from the Alaska Whale Foundation
3 from INTEL.

By end of day yesterday we had not collected a single snot sample.  Today the live rehearsal started at 4:00 and at about 5:30 pm NYC was going to cut to us to join a drone live over a whale (WHAAAT?).  We had a successful snot collection earlier in the day but the whale swam out of our satellite coverage area.  For the next five hours we did not see a blow.  Then we saw a solitary whale that we followed for over an hour that would surface and blow once, yes once – I was losing my mind (as I think was the director Alex).

The Inspire 2 drones have about a 30 min flight time so about 10 min before we were to go live Christian and I launched our drones with little hope of getting more than nice water and Alaska shots from the air.  Two minutes before we were to go live I managed to get over our single blow whale (too high to collect snot) but I could see it though the water and it was not diving deep so… I was able to stay with it as it swam just under the water, we went live and it surfaced and we got the blow.  Unblooming believable!!!!!! the only scary thing now is we have to do better than this tomorrow for the real live show (8:00 Eastern Time Nat Geo channel).

The spectacularly good news is that the INTEL team hit it out of the ball park with regards to the volumetric’s and real time photo ID data project that they have been working on.  I have attached two images, both of which represent (to the best of my knowledge) firsts in the field of whale biology.  The first image shows the automatic volumentics measurement that was conducted in flight over a whale, in real time (note the drone just above and to the left).  The second shows a whale whose identity was confirmed before the drone made it back to the boat. Just amazing, I am so tired right now I cannot give this Ted, Bryn & Javier the appropriate credit for what they have pulled off in such a short time but we will do a longer blog on this after the live TV show.

 

Alaska has tried to make up for the lack of whales with other animal abundances, Christian Miller has of course caught many of these with his amazing camera skills, alas the internet here is so slow I can only attach very low res files.

Thank you again to our friends at Parley for the Oceans for working with us to make this the incredible project that it is. Thank you also to the  23 people in the one of a kind Alaska SnotBot team.

I ask all of you to pray to the whale gods for us, we pulled off what I think is a million to one shot today and we don’t want to work to these odds tomorrow.

Hoping to get some sleep tomorrow night after successful Snot collection on live TV!!!!!

Snotless in Alaska (for one more day).

Iain

SnotBot Expedition IV, Mexico 2017: Lights, cameras, action

By | Ocean Alliance News | No Comments

Dear Friends,

Roger Payne set the stage at Ocean Alliance many years ago, insisting that there should always be a strong education component of every scientific endeavor we are involved in. More than ever before it is vitally important for scientists to effectively communicate to the general public what they are doing, how they are doing it, and most importantly, why they are doing it.  We are very lucky to have a Nutopia documentary team with us in the Sea of Cortez filming the first SnotBot expedition of 2017. Nutopia is a British production company that is making a series of environmental shows for a major US TV network (more on that as we are closer to the release date).

The SnotBot 2017 Sea of Cortez team.

The SnotBot 2017 Sea of Cortez team.

Truth be said it is almost impossible to do science and shoot a documentary at the same time; both efforts take a lot of focus and involve a lot of equipment, so we have had to compromise on our scientific goals somewhat. Luckily for us the Nutopia folks have been a real pleasure to work with. They have told us that this production will be more stylized, and that has meant that more effort has gone into every shot so that they can tell a powerful visual story. We have seen that with the mass of camera gear they brought down and the many different angles from which they have shot every activity.

The Nutopia crew brought lots of gear, including this gyro-stabilized camera.

The Nutopia crew brought lots of gear, including this gyro-stabilized camera.

We have been working out of two small boats (approx. 26 feet) one for the science team and one for the documentary team, although people seem to be constantly changing between boats during the day. Nutopia has a team of six people (plus the local boat captain), and the SnotBot team is five plus Michael Fishbach from the Great Whale Conservancy and our amazing boat captain Alberto, so 14 people in all.

Clearly each team is determined to have their project succeed: we want the data and they want the shot. On top of this there never seems to be enough time – when you balance our potentially optimistic goals against weather delays, uncooperative, or even absent animals and the constant logistical challenges, it means that we are lucky if the day only runs from dawn to dusk (and when we get back to our accommodation we have to process samples and back up our images and flight data).

Certainly, we will get less physical blow samples on this trip because we have had to dedicate time to our documentary team, but we believe that this is a worthwhile investment.

A blue whale blow sample on SnotBot's petri dish.

A blue whale blow sample on SnotBot’s petri dish.

Nutopia has engaged our superstar SnotBot cameraman Christian Miller; you have seen some of his photos from Alaska (and my last two posts). Apologies that some of todays photos are not as exciting as Christian’s but I thought you might like to see the other side of this expedition.

The two unexpected requirements of documentary-making are that we had to wear the same clothes all week (in the hot sun every day working on a small boat!) so that they could have continuity with the final edit, AND we were set up every morning with wireless microphones, so we had to be a lot more circumspect about our comments and conversations during the day :-).

From SnotBot 2017 in the Sea of Cortez, I wish you fair winds and a flowing sea.

Iain

 

This work was made possible by generous support of the Waitt Foundation through a Rapid Ocean Conservation grant. It is a privilege to be supported by such a prestigious foundation, whose mission is to Restore Our Oceans to Full Productivity.

SnotBot took this image of a Southern right whale mother and calf during an expedition in Patagonia.

2016: the year in review

By | Ocean Alliance News | No Comments

A message from CEO Dr. Iain Kerr

2016 has been a most productive year for Ocean Alliance. If you go to our website (www.whale.org) you will find blogs and videos about our work with: SnotBot, STEAM initiatives, right whales, blue whales, ocean plastics and climate change. You will also find expressions of interest in our work from Secretary of State John Kerry as well as from students of the Parley Ocean School in Jamaica.

You know what a solid bang for the buck we provide: when I told commercial drone operators at the Drone World Expo the cost of a SnotBot expedition they said they didn’t know whether to laugh or cry, given how much we do for so little.

Most charities raise up to 40% of their funding in November and December; we hope that you will confirm your continued belief in the work we do by making a donation or buying some SnotBot swag in our store or adopting a whale.

From continued development of benign research tools such as SnotBot; data collection for the 48th consecutive season of our Southern right whale program (the longest continuous study of a great whale behavior on the planet) and our STEM and STEAM education initiatives. Our vision is an ocean that is healthy for whales and humans.

Please help us turn that dream into a reality.

Highlights from 2016

• January: Extended SnotBot interview with Sir Patrick Stewart posted on our website.

• February: CEO Iain Kerr was asked to join the Advisory Board of Drone World Expo, where “thought leaders, industry experts and end-users gather in the heart of Silicon Valley to present real-world solutions to business and environmental challenges.”

• March 25-April 6: Our second SnotBot Expedition was launched to Baja California. Our team visited two different locations, collecting respiratory or “blow” samples from gray whales in San Ignacio Lagoon and blue and humpback whales in Bahia La Paz, where we were visited by a film crew from drone manufacturer DJI. The expedition was an enormous success.

Ocean Alliance Mexico expedition. Photo: Christian Miller

Ocean Alliance Mexico expedition. Photo: Christian Miller

• April: Ocean Alliance was a consultant for Sonic Sea, a documentary on sound pollution in our oceans. We also provided whale recordings for Sonic Seas.

• May 30: The 4-minute video produced by DJI during our SnotBot Baja California expedition went live. This incredibly well made video, created by award winning cameraman Tom Fitz and producer Adrienne Hall, has already been viewed more than 130,000 times on YouTube.

• June 4: At the International Whaling Commission meeting in Slovenia. Dr. Mariano Sironi, one of our partners at the Instituto de Conservacion de Ballenas (ICB) in Argentina presented a report about our Southern right whale program.

• June 8: SnotBot is labelled one of the top 8 breakthroughs saving our oceans by Matthew Mulrennan, the manager of the XPRIZE Ocean Initiative.

• June 15: The Big G Foundation supported the development of EarBot. This gave us a prototype to take to Alaska during our third SnotBot Expedition for trials.

• June 18: Ocean Alliance collaborated with the North Shore Arts Association for an exhibit and fundraiser that lasted over a month. This show included a series of talks including one by Iain Kerr and a performance of “Sea Change” by Roger Payne and Lisa Harrow. Local artists painted on old slates from one of the Paint Factory buildings; these unique and historical pieces of art were then put up for auction. Over $9,000 was raised and shared between Ocean Alliance and NSAA.

Painting on slate tile by Anne Demeter

Painting on slate tile by Anne Demeter

• July 22: We were given a 30-foot Bertram vessel called Double Header. This is the perfect boat for our regional SnotBot and toxicology programs. We have named the vessel Cachalot, after the French/Spanish name for sperm whale.

• July 26: We hosted a successful Harbor Cruise fundraiser with the generous help of our partners at Seven Seas Whale Watch and their vessel the Privateer IV. We raised over $4,000 which, for a first event, is a success. We intend to make this an annual event. http://www.whale.org/gloucester-harbor-cruise/

• August 10: Ocean Alliance launched —this one to Frederick Sound, Alaska. third SnotBot expedition —this one to Frederick Sound, Alaska. The trip achieved solid advances in SnotBot methodology and we were able to collect significantly more samples than our original goal. We also made the first tests of ‘EarBot’ (for listening to whales underwater) and ‘FLIRBot’ (a drone equipped with an InfraRed camera for studying/detecting whales at night/in lowlight conditions) and collected some exceptional footage of whales.

• Stories on SnotBot exceeded 400 press articles worldwide! This includes two facebook videos that combined have gotten over 2 MILLION views!
https://www.facebook.com/PlayGroundMag/videos/1228292177210656/
https://www.facebook.com/thedodosite/videos/911934465607896/

• August: SnotBot is to be a kids book! We are working with the publishers Houghton Mifflin to create a “high interest reading” book for Pre-K through Grade 6, that will be released in 2017! The book is being produced as part of a Pinnell Classroom Literacy project, a high-interest reading and literacy program.

• September 3 – 5: we held an open house and art exhibition on our site at the Gloucester Paint Manufactory in association with the Trident Gallery, called The Deep Sea Has Its Stars. Over 1,000 people visited the site, which was wonderful exposure for us in the city of Gloucester, and we raised over $4,000. The success of this event means that we will also make it an annual event. Board members Linde McNamara and her husband, Mac, volunteered at the event.

• September 7: Roger Payne joined Paul Winter in Nantucket for a “Whales Alive” concert.

• September 11: John Atkinson flew down to Argentina to join Vicky Rowntree and the ICB team for the 47 consecutive field season and aerial survey studying the Southern right whales at Peninsula Valdez.

• September 12: Tom Costello from NBC Nightly News visited Ocean Alliance to shoot a story on SnotBot. http://www.whale.org/nbc-nightly-news-visits-ocean-alliance/

• September 16 – 17: Roger Payne and Andy Rogan attended the Our Oceans Conference in Washington D.C., hosted by the State Department. SnotBot was personally invited to be at the event. Roger and Andy introduced SnotBot to Secretary of State John Kerry, who labelled footage of whales filmed from SnotBot ‘mesmerizing’ and ‘so amazing.’ The conference also included keynote speeches by President Barack Obama and Leonardo DiCaprio. Other exhibition hall presenters included Oceana, SkyTruth/Google, Pew, the US Navy, Seavision, Liquid Robotics, and NASA (all with multimillion-dollar budgest as against Ocean Alliance’s less than $1 million budget).

Roger Payne and Andy Rogan introduce John and Teresa Kerry to SnotBot

Roger Payne and Andy Rogan introduce John and Teresa Kerry to SnotBot

• September 20: SnotBot is announced as one of three winners of the American Institute of Aeronautics and Astronautics (AIAA) Innovative Drone Exploration and Application (IDEA) Competition. Award to be presented at Drone World Expo.

• September: Roger Payne’s work was featured in the 2016 Indianapolis Prize Guide to Animal Conservation Giving.
Pages 32 & 33.

• October 14 – 16: Iain Kerr worked with the Parley Ocean School in Jamaica to educate and introduce disadvantaged children to our oceans and the problem of ocean plastics.

• November 7 – 14: Iain Kerr went to the Maldives with Adidas and Parley for the Oceans in a program associated with the new Adidas shoe made from recycled ocean plastic.

Iain and other Parley Ocean School attendees took more than 60 Maldivian school kids out on a reef to snorkel -- some had never snorkeled before.

Iain and other Parley Ocean School attendees took more than 60 Maldivian school kids out on a reef to snorkel — some had never snorkeled before.

• October 14: Our work with the local high school robotics team, the Ipswich Tigers, is reported on by CBS Boston affiliate WBZ-TV. The students made an altimeter for SnotBot that informs the pilot how high the drone is above the whale through an earpiece, a vital piece of equipment. This is the kind of program we love: high school kids making a genuine difference to our primary research program. This was also reported in our local newspaper.

• November 1: Of all the videos made on SnotBot, this is one of the most exciting! The US Department of State made a video on the program that they then shared with their embassies all around the world!

• November 15 – 16: Iain Kerr gave a SnotBot talk at the Drone World Expo in San Jose, and gave a second SnotBot talk as a winner of the American Institute of Aeronautics and Astronautics (AIAA) Innovative Drone Exploration and Application (IDEA) Competition.

SnotBot makes a splash at Drone World Expo

“It has been an incredibly productive year! We thank you for giving us the opportunity to do what we do best. As we look to 2017 I can tell you that we have some very exciting projects and collaborations on the horizon! We’re excited to take SnotBot to the next level, and formally introduce EarBot, FLIRBot, and other bot’ that are changing the way we do whale research. We’re also looking forward to continuing renovations of our home at the Gloucester Paint Manufactory and our 48th consecutive Southern right whale season. But most of all we are looking forward to sharing some adventures with all of you and continuing to take steps towards protecting whales and the ocean environment in which they swim.
Roger Payne – Founder and President.