amy + gcp   332

Google App Engine can provide an easy way to get a persistent Tensorboard server with authentication for a small cost.

In addition if you find yourself pulling a large amount of data from GCS when starting up Tensorboard servers, you may actually pay less for a persistent GAE server, since you don't pay for data egress between GCS and GAE.
gae  gcp  TensorFlow  machine_learning 
22 days ago by amy
Machine Learning with Scikit-Learn and Xgboost on Google Cloud Platform (Cloud Next '18) - YouTube
If you're doing Machine Learning in Python, you're probably familiar with open source frameworks like scikit-learn and XGBoost. But are you using Google Cloud Platform to speed up your training, scale your prediction and deepen your understanding of unstructured data? This talk will provide practical tips for developers and data scientists to make the most of GCP. We'll discuss the Machine Learning APIs, AutoML, Kubeflow, Google Compute Engine, and Cloud ML Engine. We'll show you what to consider when choosing between these services, and help you get started quickly.
gcp  cmle  machine_learning 
10 weeks ago by amy
Virtual Private Cloud (VPC)  |  Google Cloud
Virtual Private Cloud (VPC) gives you the flexibility to scale and control how workloads connect regionally and globally. When you connect your on-premises or remote resources to GCP, you’ll have global access to your VPCs without needing to replicate connectivity or administrative policies in each region.
gcp  security  privacy 
11 weeks ago by amy
How Geotab Drives Smart City Innovation, Using BigQuery (Cloud Next '18) - YouTube
BQ GIS intro from Chad

Geotab is one of the world's leading asset tracking companies, with more than 1M vehicles under service every day. They use BigQuery as their enterprise data warehouse to derive new insights from their location data – opening up new possibilities of value for their customers and entirely new lines of business.
gcp  bigquery 
11 weeks ago by amy
Building the World's Largest Enterprise Data Warehouse with BigQuery (Cloud Next '18) - YouTube
w/ Jordan Tigani

This talk, by one of the founders of the BigQuery team and a founder and current CTO of Looker, will make the case that BigQuery is not just another Enterprise Data Warehouse. It will show how BigQuery's unique properties follow from its technical architecture. The talk will announce new performance, security, cost, and manageability features. It will also show tricks and tips for getting more out of BigQuery. And finally it will show you how to push BigQuery to do things that you never thought were possible in a Data Warehouse.
gcp  bigquery 
11 weeks ago by amy
This repository contains example Kubernetes applications ("apps") that meet the requirements for integration with Google Cloud Marketplace. For a complete description of those requirements, see the technical onboarding guide. TODO: add link

The related marketplace-k8s-app-tools repository contains utilities for testing the integration of an app with Marketplace, including a test harness for simulating UI-based deployment. The repository is submoduled under /vendor/marketplace-tools.
gcp  kubernetes  k8s 
11 weeks ago by amy
Kubernetes-based platform to build, deploy, and manage modern serverless workloads
gcp  google  serverless  kubernetes  k8s 
11 weeks ago by amy
Making Kubernetes IP addresses static on Google Container Engine | Terrence Ryan
I’ve been giving a talk and demo about Kubernetes for a few months now, and during my demo, I have to wait for an ephemeral, external IP address from a load balancer to show off that Kubernetes does in fact work.  Consequently, I get asked “Is there any way to have a static address so that you can actually point a hostname at it?” The answer is: of course you can.
gke  kubernetes  k8s  gcp 
11 weeks ago by amy
training-data-analyst/serving_embed.ipynb at master · GoogleCloudPlatform/training-data-analyst
Serving embeddings
This notebook illustrates how to:

- Create a custom embedding as part of a regression/classification model
- Representing categorical variables in different ways
- Math with feature columns
- Serve out the embedding, as well as the original model's predictions
TensorFlow  machine_learning  gcp 
july 2018 by amy
Give meaning to 100 billion analytics events a day – Teads Engineering – Medium
In this article, we describe how we orchestrate Kafka, Dataflow and BigQuery together to ingest and transform a large stream of events. When adding scale and latency constraints, reconciling and reordering them becomes a challenge, here is how we tackle it.
gcp  bigquery  dataflow 
may 2018 by amy
Different bash sessions for different gcloud users/projects
Different bash sessions for different gcloud users/projects

from cbro
gcloud  gcp 
april 2018 by amy
This project implements a gRPC server that satisfies the Cloud Pub/Sub API as an emulation layer on top of an existing Kafka cluster configuration. The emulator is exposed as a standalone Java application with a mandatory configuration file passed as an argument at runtime. It is fully compatible with the latest versions of the Google Cloud Client libraries and is explicitly tested against the Java version.
google  oss  gcp  pubsub 
april 2018 by amy
googlegenomics/gcp-variant-transforms: GCP Variant Transforms
This is a tool for transforming and processing VCF files in a scalable manner based on Apache Beam using Dataflow on Google Cloud Platform.

It can be used to directly load VCF files to BigQuery supporting hundreds of thousands of files, millions of samples, and billions of records.
genomics  gcp  bigquery 
march 2018 by amy
Machine Learning with TensorFlow on Google Cloud Platform: code samples
new courses!: “Machine Learning with TensorFlow on Google Cloud Platform: code samples”

via @lak_gcp
TensorFlow  machine_learning  gcp 
february 2018 by amy
Shared VPC Overview  |  VPC  |  Google Cloud Platform
You can share Google Cloud Platform (GCP) Virtual Private Cloud (VPC) networks across projects in your Cloud Organization. This capability is referred to as Shared VPC.

Note: Shared VPC was previously known as Cross-Project Networking (XPN).
In large organizations, you may need to put different departments or different applications into different projects for purposes of separating budgeting, access control, and so on. With Shared VPC, Cloud Organization administrators can give multiple projects permission to use a single, shared VPC network and corresponding networking resources.

Shared VPC allows creation of a VPC network of RFC1918 IP spaces that associated projects can then use. Admins in associated projects can create virtual machine (VM) instances in the shared VPC network spaces. Network and security admins can create VPNs and firewall rules usable by all the projects in the VPC network. Consistent policies can be applied and enforced easily across a Cloud Organization.
february 2018 by amy
Forbes: 12 Amazing Deep Learning Breakthroughs of 2017
1. DeepMind’s AlphaZero Clobbered The Top AI Champions In Go, Shogi, And Chess
2. OpenAI’s Universe Gained Traction With High-Profile Partners
3. Sonnet & Tensorflow Eager Joined Their Fellow Open-Source Frameworks
4. Facebook & Microsoft Joined Forces To Enable AI Framework Interoperability
5. Unity Enabled Developers To Easily Build Intelligent Agents In Games
6. Machine Learning As A Service (MLAAS) Platforms Sprout Up Everywhere
7. The Gan Zoo Continued To Grow
8. Who Needs Recurrence Or Convolution When You Have Attention? (Transformer)
9. AutoML Simplified The Lives Of Data Scientists & Machine Learning Engineers
10. Hinton Declared Backprop Dead, Finally Dropped His Capsule Networks
11. Quantum & Optical Computing Entered The AI Hardware Wars
12. Ethics & Fairness Of ML Systems Took Center Stage
machine_learning  TensorFlow  google  gcp 
february 2018 by amy
Google Cloud Platform Blog: Introducing faster GPUs for Google Compute Engine
Today, we're happy to make some massively parallel announcements for Cloud GPUs. First, Google Cloud Platform (GCP) gets another performance boost with the public launch of NVIDIA P100 GPUs in beta. Second, NVIDIA K80 GPUs are now generally available on Google Compute Engine. Third, we're happy to announce the introduction of sustained use discounts on both the K80 and P100 GPUs.
Cloud GPUs can accelerate your workloads including machine learning training and inference, geophysical data processing, simulation, seismic analysis, molecular modeling, genomics and many more high performance compute use cases.
The NVIDIA Tesla P100 is the state of the art of GPU technology. Based on the Pascal GPU architecture, you can increase throughput with fewer instances while saving money. P100 GPUs can accelerate your workloads by up to 10x compared to K801.
gcp  machine_learning  TensorFlow  cmle 
september 2017 by amy
RT : Familiar with AWS but want to learn ? talks about our latest course
GCPtraining  GCP  from twitter
september 2017 by amy
Multistyle Pastiche Generator
Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur have extended image style transfer by creating a single network which performs more than one stylization of an image. The paper[1] has also been summarized in a Google Research Blog post. The source code and trained models behind the paper are being released here.
TensorFlow  machine_learning  magenta  gcp 
august 2017 by amy
NYTimes/marvin: A go-kit HTTP server for the App Engine Standard Environment
Marvin is a go-kit server for Google App Engine

Marvin + GAE -> let's get it oooonnnn!

:insert adorable Marvin Gaye inspired gopher here:

Marvin provides common tools and structure for services being built on Google App Engine by leaning heavily on the go-kit/kit/transport/http package. The service interface here is very similar to the service interface in NYT's gizmo/server/kit package so teams can build very similar looking software but use vasty different styles of infrastructure.

Marvin has been built to work with Go 1.8, currently in open beta on App Engine Standard. Use it by setting api_version: go1.8 in your app.yaml.
gae  google  gcp  gokit  golang 
august 2017 by amy
Moving The New York Times Games Platform to Google App Engine
RT @iRomin: 3-person team running @nytimes Crossword site, running costs slashed to half, powered by AppEngine + Go and more.
gcp  gae  golang 
august 2017 by amy
Modeled after the Broad Institute’s Firehose analysis infrastructure, FireCloud democratizes data access and facilitates collaboration by providing a robust, scalable platform accessible to the community at large. Using the elastic compute capacity of Google Cloud, FireCloud empowers analysts, tool developers and production managers to perform large-scale analysis, engage in data curation, and store or publish results.
genomics  cloud  health  medicine  bigdata  gcp 
july 2017 by amy
ZEIT – Universal Now: Now, on Every Cloud
deploy cross-cloud with one command
aws  cloud  gcp  nodejs 
july 2017 by amy
This is an example application demonstrating how TensorFlow Object Detection API and pretrained models can be used to create a general object detection service.
TensorFlow  machine_learning  gcp 
july 2017 by amy
Node Auto-Repair on Container Engine  |  Container Engine  |  Google Cloud Platform
Container Engine's Node Auto-Repair feature helps you keep the nodes in your cluster in a healthy, running state. When enabled, Container Engine makes periodic checks on the health state of each node in your cluster. If a node fails consecutive health checks over an extended time period (approximately 10 minutes), Container Engine initiates a repair process for that node.

If Container Engine detects that a node requires repair, that node will first be drained, and then Container Engine will re-create the node VM. The drain might not succeed if the node is unresponsive or is too unhealthy to process the drain command.

If multiple nodes require repair, Container Engine repairs one node at a time, with each repair lasting approximately 5-10 minutes. If you disable node auto-repair at any time during the repair process, the in-progress repairs are not cancelled and will still complete for any node currently under repair.

Container Engine will generate an entry in its operation logs for any automated repair event. You can check the logs by using the gcloud container operations list command.
kubernetes  k8s  gke  gcp 
july 2017 by amy
BQ  |  Google Cloud Platform
To scale quickly, BQ began migrating services to Docker containers running on Google Container Engine--built on the open source Kubernetes system. With Container Engine, BQ has a managed orchestration platform for Kubernetes to keep its small team focused on innovation rather than IT maintenance.

“Since we moved to Google Container Engine on Google Cloud Platform, our development is much easier. There is no extra work for our team in having to manage our extensive environment, allowing us to ship faster and to focus on more strategic projects.” - Pablo Moncada, IT DevOps Team Lead, BQ
By using preemptible (highly affordable, short-lived) Google Compute Engine instances with automated container management for fault tolerance, BQ gets the best of both worlds: highly affordable compute instances and high availability. It can therefore operate at a significantly lower cost and provide better service to customers and employees. Instead of running each cloud service on a separate node, BQ now runs 350 services on just 20 nodes.
gke  kubernetes  gcp 
july 2017 by amy
« earlier      
per page:    204080120160

Copy this bookmark: