本文整理了Java中org.kitesdk.data.Dataset.getNamespace()
方法的一些代码示例,展示了Dataset.getNamespace()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Dataset.getNamespace()
方法的具体详情如下:
包路径:org.kitesdk.data.Dataset
类名称:Dataset
方法名:getNamespace
[英]Get the namespace that contains this Dataset. A namespace is a string that identifies a logical group of datasets.
[中]获取包含此数据集的命名空间。命名空间是标识数据集逻辑组的字符串。
代码示例来源:origin: org.kitesdk/kite-tools
private URI getLegacyRepoUri(Dataset<GenericRecord> dataset) {
return FlumeConfigCommand.this.getLegacyRepoUri(dataset.getUri(), dataset.getNamespace());
}
代码示例来源:origin: kite-sdk/kite
private static DatasetRepository getDatasetRepository(JobContext jobContext) {
Configuration conf = Hadoop.JobContext.getConfiguration.invoke(jobContext);
DatasetRepository repo = DatasetRepositories.repositoryFor(conf.get(KITE_OUTPUT_URI));
if (repo instanceof TemporaryDatasetRepositoryAccessor) {
Dataset<Object> dataset = load(jobContext).getDataset();
String namespace = dataset.getNamespace();
repo = ((TemporaryDatasetRepositoryAccessor) repo)
.getTemporaryRepository(namespace, getJobDatasetName(jobContext));
}
return repo;
}
代码示例来源:origin: org.kitesdk/kite-data-mapreduce
private static DatasetRepository getDatasetRepository(JobContext jobContext) {
Configuration conf = Hadoop.JobContext.getConfiguration.invoke(jobContext);
DatasetRepository repo = DatasetRepositories.repositoryFor(conf.get(KITE_OUTPUT_URI));
if (repo instanceof TemporaryDatasetRepositoryAccessor) {
Dataset<Object> dataset = load(jobContext).getDataset();
String namespace = dataset.getNamespace();
repo = ((TemporaryDatasetRepositoryAccessor) repo)
.getTemporaryRepository(namespace, getJobDatasetName(jobContext));
}
return repo;
}
代码示例来源:origin: org.kitesdk/kite-tools
target.getDataset().getNamespace(),
UUID.randomUUID().toString());
代码示例来源:origin: org.kitesdk/kite-tools
target.getDataset().getNamespace(),
UUID.randomUUID().toString());
代码示例来源:origin: org.kitesdk/kite-tools
target.getDataset().getNamespace(),
UUID.randomUUID().toString());
代码示例来源:origin: kite-sdk/kite
boolean deleteAllUnsafe(boolean useTrash) {
boolean deleted = false;
if (dataset.getDescriptor().isPartitioned()) {
for (StorageKey key : partitionIterator()) {
deleted = (useTrash ? FileSystemUtil.cleanlyMoveToTrash(fs, root, key.getPath())
: FileSystemUtil.cleanlyDelete(fs, root, key.getPath())) || deleted;
if (listener != null) {
// the relative path is the partition name, so we can simply delete it
// in Hive
listener.partitionDeleted(dataset.getNamespace(),
dataset.getName(), key.getPath().toString());
}
}
}
else {
for (Path path : pathIterator()) {
deleted = (useTrash ? FileSystemUtil.cleanlyMoveToTrash(fs, root, path)
: FileSystemUtil.cleanlyDelete(fs, root, path)) || deleted;
}
}
return deleted;
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testRelative() {
DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:target/data");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Record> ds = Datasets.<Record, Dataset<Record>>
load("dataset:file:target/data/ns/test", Record.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Path cwd = localFS.makeQualified(new Path("."));
Assert.assertEquals("Locations should match",
new Path(cwd, "target/data/ns/test").toUri(), ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testRelative() {
DatasetRepository repo = DatasetRepositories
.repositoryFor("repo:hdfs://" + hdfsAuth + "/data?absolute=false");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Object> ds = Datasets.<Object, Dataset<Object>>
load("dataset:hdfs://" + hdfsAuth + "/data/ns/test?absolute=false",
Object.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Path cwd = getDFS().makeQualified(new Path("."));
Assert.assertEquals("Locations should match",
new Path(cwd, "data/ns/test").toUri(), ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testAbsolute() {
DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:/tmp/data");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Record> ds = Datasets.<Record, Dataset<Record>>
load("dataset:file:/tmp/data/ns/test", Record.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Assert.assertEquals("Locations should match",
URI.create("file:/tmp/data/ns/test"),
ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testAbsolute() {
DatasetRepository repo = DatasetRepositories
.repositoryFor("repo:hdfs://" + hdfsAuth + "/tmp/data");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Object> ds = Datasets.<Object, Dataset<Object>>
load("dataset:hdfs://" + hdfsAuth + "/tmp/data/ns/test", Object.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Assert.assertEquals("Locations should match",
URI.create("hdfs://" + hdfsAuth + "/tmp/data/ns/test"),
ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testAbsoluteTrailingSlash() {
DatasetRepository repo = DatasetRepositories
.repositoryFor("repo:hdfs://" + hdfsAuth + "/tmp/data/");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Object> ds = Datasets.<Object, Dataset<Object>>
load("dataset:hdfs://" + hdfsAuth + "/tmp/data/ns/test/", Object.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Assert.assertEquals("Locations should match",
URI.create("hdfs://" + hdfsAuth + "/tmp/data/ns/test"),
ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testAbsoluteWebHdfs() {
Assume.assumeTrue(!Hadoop.isHadoop1());
String webhdfsAuth = getConfiguration().get(
DFSConfigKeys.DFS_NAMENODE_HTTP_ADDRESS_KEY);
DatasetRepository repo = DatasetRepositories
.repositoryFor("repo:webhdfs://" + webhdfsAuth + "/tmp/data");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Object> ds = Datasets.<Object, Dataset<Object>>
load("dataset:webhdfs://" + webhdfsAuth + "/tmp/data/ns/test", Object.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Assert.assertEquals("Locations should match",
URI.create("webhdfs://" + webhdfsAuth + "/tmp/data/ns/test"),
ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testAbsoluteRoot() {
DatasetRepository repo = DatasetRepositories
.repositoryFor("repo:hdfs://" + hdfsAuth + "/");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
Dataset<Object> ds = Datasets.<Object, Dataset<Object>>
load("dataset:hdfs://" + hdfsAuth + "/ns/test",
Object.class);
Assert.assertNotNull("Should load dataset", ds);
Assert.assertTrue(ds instanceof FileSystemDataset);
Assert.assertEquals("Locations should match",
URI.create("hdfs://" + hdfsAuth + "/ns/test"),
ds.getDescriptor().getLocation());
Assert.assertEquals("Descriptors should match",
repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", ds.getNamespace());
Assert.assertEquals("Should report correct name",
"test", ds.getName());
repo.delete("ns", "test");
}
代码示例来源:origin: kite-sdk/kite
@Test
public void testViewConstraints() {
DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:/tmp/data");
repo.delete("ns", "test");
repo.create("ns", "test", descriptor);
RefinableView<Record> v = Datasets.<Record, RefinableView<Record>>
load("view:file:/tmp/data/ns/test?username=user", Record.class);
Assert.assertNotNull("Should load view", v);
Assert.assertTrue(v instanceof FileSystemView);
Assert.assertEquals("Locations should match",
URI.create("file:/tmp/data/ns/test"),
v.getDataset().getDescriptor().getLocation());
DatasetDescriptor loaded = repo.load("ns", "test").getDescriptor();
Assert.assertEquals("Descriptors should match",
loaded, v.getDataset().getDescriptor());
Assert.assertEquals("Should report correct namespace",
"ns", v.getDataset().getNamespace());
Assert.assertEquals("Should report correct name",
"test", v.getDataset().getName());
Constraints withUser = new Constraints(loaded.getSchema())
.with("username", new Utf8("user"));
Assert.assertEquals("Constraints should be username=user",
withUser, ((FileSystemView) v).getConstraints());
repo.delete("ns", "test");
}
内容来源于网络,如有侵权,请联系作者删除!